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I.    INTRODUCTION 

A.         MOTIVATION 

The  advances  in  computer  technology  in  the  last  decade  have  made  available  the 
tools  to  accomplish  certain  missions  that  before  that  time  would  have  required  much 
greater  involvement  of  programmers,  technicians  and  other  people  with  specialized 
knowledge.  These  advances  have  also  made  the  solution  to  some  problems  more 
economically  viable.  Decision  Support  Systems  (DSS)  have  become  more  useful  and 
accessible  to  managers  because  of  the  wider  availability  of  more  economical  hardware 
and  software.  The  computer's  ability  to  organize  and  present  data  for  a  manager  to 
analyze  and  the  increase  in  the  capacity  of  desktop  computers,  make  them  an  ideal  tool 
and/or  element  in  a  Decision  Support  System. 

This  thesis  will  address  DSS  as  used  in  a  large-scale  procurement  decisiormiaking 
process.  It  will  address  military  procurement  and  the  factors  and  information  that  the  DSS 
should  provide  to  the  decisionmaker.  Some  of  these  factors  are  costs,  availability, 
capabilities,  maintainability  and  other  factors  which  could  be  considered  political.  To  be 
usefiil  the  DSS  should  have  in-depth  and  broad  capability  to  present  the  results  of  data 
collection,  budget  information  and  other  input  variables  that  the  decisionmaker  would  in 
fact  use  to  make  these  procurement  decisions  manually. 

The  decisionmaking  process  in  any  large-scale  organization  has  many  factors  that 
may  directly  or  indirectly  affect  any  given  decision.  Some  factors  which  in  reality  should 
not  effect  the  decision  may  nevertheless  be  used  in  the  decisionmaking  process  and  lead 
to  a  bad  or  costly  decision.  This  process  on  the  surface  may  appear  to  be  simple.  In 
reality,  it  is  very  complicated  and  a  decisionmaker  cannot  be  expected  to  reach  a  perfect 
decision.  Complicating  this  are  the  normal  human  factors  in  any  organization:  negligence, 
simple  mistakes,  or  malfeasance.  These  factors  could  lead  to  loss  of  money  or  property, 
endanger  human  life,  or  seriously  undermine  the  nation's  ability  to  defend  itself. 


A  goal  of  any  procurement  system  is  to  reduce  the  possibility  of  mistaken 
decisions  or  errors  in  procurement.  I  believe  a  DSS  would  be  useful  to  the  Republic  of 
China,  to  reduce  costs  and  help  decisionmakers  make  more  accurate  and  timely 
procurement  decisions. 

The  Republic  of  China  has  reached  a  point  in  time  where  it  is  necessary  to  replace 
or  upgrade  obsolete  military  equipment.  The  international  situation  and  the  needs  of 
Republic  of  China  require  this  process  to  be  accelerated  if  we  are  to  accomplish  our  goals 
of  self-reliance  and  increase  our  combat  capabilities.  A  sound  procurement  system  based 
on  accurate  data  collection  and  a  DSS  would  serve  as  a  much  better  basis  for  procurement 
than  procedures  used  in  the  recent  past. 

The  nature  of  military  procurement  systems  is  that  decisions  are  more  likely  to  be 
made  in  the  political  sphere  than  in  the  more  practical  sphere  of  military  requirements. 
This  process  could  ultimately  lead  to  the  Republic  of  China  being  unable  to  defend  itself. 
The  establishment  of  a  DSS  would  lend  to  a  procurement  system  as  an  institutionalized 
decisionmaking  process,  less  influenced  by  the  political  process  and  more  resistant  to 
fraud  and  other  abuses  which  lead  to  these  scandals. 

The  quality  of  decisionmaking  would  also  be  improved  with  a  DSS  in  that  the 
system  would  present  all  the  factors  needed  by  the  decisionmaker  in  any  procurement 
decision.  To  present  these  factors  an  effective  data  collection  system  is  necessary.  This 
data  collection  system  itself  would  minimize  bad  decisions. 

B.    BACKGROUND 

I  have  spent  a  great  deal  of  time  in  the  study  of  procurement  regulations  and  other 
documents  related  to  procurement.  The  military  procurement  system  is  Byzantine  in 
nature,  changes  from  moment  to  moment  and  defies  any  attempt  at  simplification.  I  have 
also  interviewed  staff  personnel  involved  in  military  procurement.  The  purpose  of  this 
study  and  interviews  was  to  identify  the  problems  and  factors  which  should  be  presented 
by  any  DSS  system.  The  following  are  initial  findings: 


Procurement  is  a  Multi-Criteria  Issue.  Factors  such  as  cost,  longer  range, 
greater  capabilities,  or  easier  maintenance,  must  be  considered  together  to 
reach  a  decision.  This  means  that  a  single  factor,  cost  for  instance,  carmot  be 
used  to  make  the  decision. 

Dissimilar  Selection  Criteria.  Sound  procurement  decisions  rest  on  a  wide 
variety  of  often  dissimilar  criteria,  including  various  cost  and  performance 
criteria,  risks,  political  interests  and  a  number  of  support  criteria  affected  by 
all  of  the  above.  The  criteria  falls  into  two  basic  categories:  qualitative  and 
quantitative.  Quantitative  criteria,  such  as  the  various  costs,  can  often  be 
directly  compared  with  one  another.  However,  dissimilar  quantitative  and 
qualitative  criteria,  such  as  procurement  cost  and  degree  of  risk,  cannot  be 
easily  compared.  A  DSS  can  help  address  this  problem  of  reconciling 
dissimilar  criteria. 

Procurement  Time.  Each  procurement  regardless  of  size  requires  a  certain 
minimum  of  time  for  decisionmaking  and  decision  implementation.  This  time 
is  required  for  research  and  study  of  the  factors  and  the  effects  of  the  decision. 

Political  Consideration.  The  political  situation  is  more  complex  than  ever 
before.  It  is  difficult  to  find  a  decision  that  can  be  satisfactory  to  all  parties. 
Uimecessary  complications  and  limitations  are  generated,  and  even  worse, 
some  political  scandals  emerge,  such  as  corruption,  hidden  agendas  and 
favoritism. 

Conflicting  Interests.  In  large  systems  there  are  many  stakeholders  and  it  is 
inevitable  that  there  will  be  conflicting  interests.  It  may  be  impossible  to 
equitably  balance  the  interests  of  every  stakeholder. 

Subjectivity.  The  objectivity  of  procurement  personnel  is  influenced  by  their 
experience  and  position  in  the  system.  Their  thinking  is  subject  to  factors  of 
career  and  acceptability  in  the  group.  Their  decisionmaking  process  caimot  be 
normalized  wdthout  the  help  of  an  institutionalized  system  of  decisionmaking. 
Military  personnel,  in  particular,  are  subject  to  changes  in  status  and  position 
which  will  have  varying  effects  on  their  morale  and  their  ability  to  make 
objective  decisions  with  regard  to  procurement. 

Time  Constraints.  A  supervisor's  time  is  usually  filled  with  urgent  problems 
and  this  leads  to  the  syndrome  of  "fighting  fires".  The  inefficiency  of  this 
approach  leads  to  the  neglect  of  the  long-term  procurement  process  and  results 
in  inefficiency  and  higher  cost.  The  supervisor's  decisionmaking  is  unduly 
affected  by  whatever  fire  he  may  currently  be  working  on.  Significant  errors 
cannot  be  avoided  because  of  the  supervisor's  short  horizon.  He  is  unlikely  to 


see  far  enough  into  the  future  to  predict  them.  Nor  is  he  Hkely  to  have  the  time 
in  study  to  make  efficient  and  correct  or  timely  decisions. 

•  Day-to-Day  Process.  Data  collection  and  analyses  in  the  procurement 
processes  are  done  repeatedly.  New  data  require  new  analyses.  The  repetitive 
nature  of  these  operations  leads  to  errors  and  lackadaisical  performance  by 
personnel  involved  in  this  task.  These  same  personnel  are  subject  to  the 
feeling  that  a  task  that  is  done  over  and  over  again  cannot  possibly  be  of  any 
importance,  since  it  will  again  have  to  be  done  next  week  or  even  tomorrow. 
This  will  lead,  in  time,  to  an  accumulation  of  errors  that  will  make  the  data 
collection  worthless. 

C.         WHY  USE  A  DECISION  SUPPORT  SYSTEM? 

A  successful  procurement  decision  depends  on  the  information  available  to  and 
influences  on  the  decisionmaker.  A  good  decision  will  improve  the  capabilities  of  the 
military  and/or  save  taxpayer's  money.  A  bad  decision  may  result  in  inefficiency  and  the 
reduced  effectiveness  of  the  Republic  of  China's  combat  forces  or  in  a  worse  case 
scenario  the  failure  of  those  forces  in  combat.  These  problems  should  be  studied  very 
carefiilly  and  the  decisions  should  have  the  best  support  systems  available.  If  a  DSS  can 
be  developed  for  the  Republic  of  China,  it  would  provide  decisionmakers  at  different 
levels  with  better  information  to  base  their  decisions. 

Computer  capabilities  and  tools  to  accomplish  our  mission  of  improving  the 
procurement  process  are  readily  available.  These  tools  would  be  used  even  if  the  DSS 
system  were  a  manual  system;  therefore,  it  would  improve  the  process  if  a  standard 
computer  system,  hardware  and  software,  were  made  a  part  of  the  design  of  the  DSS. 

I  believe  a  computer  base  DSS  would  be  the  best  solution  for  our  military 
procurement  problems.  It  would  improve  the  quality  of  decisionmaking,  providing  the 
best  recommendations  for  procurement.  In  Chapter  II,  I  will  cover  in  detail  the  type  of 
system  and  software  that  would  meet  the  Republic  of  China's  requirements. 


D. 


METHODOLOGY 


Shown  in  Figure  1  is  a  flow  chart  depicting  the  design  of  a  suitable  DSS  for 
military  procurement  decisionmaking  problems.  The  major  steps  include  the  following: 


DEFINITION 


ANALYSIS 


QUESTIONNAIRE 
DESIGN 


DECISION  CRITERIA 


ASSIGNING 
WEIGHTS 


SIMULATION 


CONCLUSION 


RECOMMENDATION 


Figure  1.  Design  Process  of  DSS 


•  Definition  -  Definition  of  the  problem.  The  definitions  of  the  goals  we  expect 
to  achieve  and  the  definitions  of  the  factors  important  to  the  decision. 

•  Analysis  -  The  characteristics  of  the  problem  are  systematically  analyzed  for 
their  influences  on  the  decision.  The  most  important  characteristics  are  used  to 
develop  the  questionnaire. 

•  Questionnaire  Design  -  The  questionnaire  is  designed  with  the  results  of  our 
definitions  and  analysis  in  mind.  It  should  also  reflect  relevant  questions  for 
the  stakeholders  involved  in  the  decision. 

•  Decision  Criteria  -  The  results  of  our  questionnaire  are  used  as  a  source  of 
statistical  data  to  develop  the  system.  With  these  results  weights  are  assigned 
to  the  various  factors.  This  should  produce  a  prioritized  list  of  factors  from 
most  important  to  least  important.  This  list  is  used  to  capture  the  most 
important  criteria  in  the  decision  model. 


• 


• 


• 


Assigning  Weights  -  Using  the  results  from  the  statistics  package,  assign 
weights  to  the  various  factors  that  are  representative  of  the  results  from  the 
user's  questionnaire. 

Simulation  -  Choose  a  small-scale  procurement  problem  as  an  example  and 
perform  a  test  with  the  software  and  the  weighted  criteria. 

Conclusion  -  Determine  the  effectiveness  of  the  DSS  by  interviewing  a  user  of 
the  system  and  weighing  the  results  against  the  users  criteria  and  probable 
decision.  The  interview  should  also  include  the  users  opinions  as  to  the 
usefiilness  of  the  system  for  making  decisions. 

Recommendation  -  Recommend  expansion  of  the  system  or  modification  of  it 
as  a  result  of  the  conclusions  reached  in  the  previous  step. 


E.         SCOPE  AND  LIMITATIONS  OF  RESEARCH 

Military  procurement  is  a  dynamic  system.  It  is  impossible  to  design  a  perfect 
solution  that  will  be  applicable  in  any  particular  scenario.  My  intent  in  this  thesis  is  to 
provide  some  solutions  that  will  be  helpful  in  making  procurement  decisions  in  a  limited 
scope  of  problems.  For  a  general  solution  to  these  problems,  a  great  deal  more  study 
would  be  required. 


Many  elements  of  military  procurement  are  classified.  It  is  therefore  difficult  to 
collect  actual  data  which  could  be  used  in  my  demonstration.  As  a  result,  I  have  created 
data  that  reflects  the  data  from  real  procurement  scenarios,  in  order  to  avoid  the  issues  of 
classification,  clearances,  and  confidentiality.  Although  the  data  I  use  is  not  confidential, 
I  believe  it  represents  a  normal  procurement  and  will  demonstrate  the  concept  outlined  in 
the  thesis.  In  Chapter  II  of  this  thesis,  I  will  discuss  the  theory  of  Decision  Support 
Systems  and  their  structure  and  how  they  differ  from  expert  systems.  I  will  also  review 
some  examples  of  Decision  Support  Systems  that  have  been  used  in  civilian  and  military 
procurement.  In  Chapter  III,  I  will  discuss  the  design  of  the  DSS  that  I  will  demonstrate 
in  this  thesis.  Chapter  III  will  consist  of  step-by-step  procedures  and  the  design  decisions 
made  for  each  step  shown  in  Figure  1 .  Chapter  IV  will  be  an  explanation  of  the  results  of 
the  simulated  procurement  problem  using  the  DSS  designed  in  Chapter  III.  In  Chapter  V  I 
will  present  my  conclusions  and  recommendations. 


II.  DECISION  SUPPORT  SYSTEM  STRUCTURE 

A.         THE  DEVELOPMENT  OF  DSS 

In  the  early  1950's  the  computers  use  was  restricted  to  scientific  and  engineering 
uses.  However,  improvements  in  technology  and  availability  moved  the  computer  from 
scientific  uses  to  business  uses.  One  of  the  first  uses  of  the  computer  in  the  commercial 
world  was  for  Transaction  Processing  Systems  (TPS).  TPS  includes  such  things  as 
payroll,  record  keeping  and  billing  systems.  Even  with  these  uses,  no  one  could  have 
predicted  the  impact  that  computers  would  have  on  business  and  management  at  the 
present  time.  Speed,  accuracy  and  storage  capacity  are  the  three  characteristics  of 
computers.  Since  that  time,  the  computer's  capacity  in  all  three  of  these  categories  has 
increased  geometrically.  These  improvements  are  the  very  reason  that  computers  are  so 
useful  as  information  processing  devices. 

There  are  five  uses  for  computers  in  management.  The  first.  Transaction 
Processing  Systems  (TPS)  is  the  most  obvious  and  still  the  most  common  use  of  large 
computer  systems.  The  next  uses  have  more  impact  on  management  decisionmaking. 
Management  Information  Systems  (MIS)  are  used  for  production  control,  sales  forecast 
and  monitoring  these  items.  Decision  Support  Systems  (DSS),  the  subject  of  this  thesis, 
are  used  for  long-range  planning  and  complex  decisionmaking.  Expert  Systems  (ES)  are 
used  for  diagnostics,  internal  controls,  planning  and  maintenance  areas  that  have  a  limited 
domain.  The  fifth  use  is  as  Executive  Information  Systems  (EIS),  these  systems  are  used 
to  support  top  management  decisionmaking  and  provide  top  management  with  reports. 
[Ref  1] 

The  classification  of  these  tools  does  not  necessarily  indicate  that  any  particular 
computer  system  is  dedicated  to  any  one  of  these  operations,  or  in  fact,  that  any  particular 
data  source  is  dedicated  to  one  management  system.  These  technologies  are  made  up  of 


three  dimensions:  a  particular  computer/hardware,  several  programs/software  and  the 
management  processes  that  gather  information  to  support  these  systems.  [Ref.  1] 
The  development  of  these  systems  are  as  follows: 

•  TPS  was  first  used  in  the  early  1950's  and  is  still  the  main  use  of  large 
computer  systems. 

•  MIS  was  first  used  in  the  1960's  by  upper  management  mostly  to  produce 
standard  operating  procedures,  decision  rules  and  reduce  cost.  This  was 
typically  done  by  replacing  clerical  personnel  who  were  employed  to  produce 
reports  with  a  computer  system.  These  systems  were  restricted  to  higher 
management  for  reasons  of  equipment  cost  as  much  as  the  utility  of  the  MIS. 

•  DSS.  In  the  1970's  Keen/Morton  coined  the  term  DSS  [Ref  2].  DSS  is  used 
to  address  semistructured  problems  and  one  of  a  kind  or  once  in  a  lifetime 
decisionmaking  situations.  This  differs  from  MIS  which  generally  supports 
only  recurring  reports  or  highly  structured  situations. 

•  ES  were  developed  in  the  1960's  as  part  of  research  in  artificial  intelligence 
(AI).  Expert  systems  have  a  limited  domain  in  that  they  are  only  applicable  to 
recurring  problems  and  are  intended  to  assist  or  replace  a  human  expert.  These 
systems  are  not  adaptable  to  new  or  unique  situations. 

From  the  above  list  we  have  a  basic  knowledge  of  management's  support  systems 
and  we  can  easily  distinguish  between  them.  Figure  2  indicates  the  relationship  between 
these  systems. 

Figure  2  is  a  notional  view  of  these  relationships.  This  view  shows  the  evolution 
of  the  use  of  data  processing  systems  in  decisionmaking,  but  does  not  represent  the  future 
development  of  management  uses  of  computers.  Figure  3  represents  the  theoretical  view 
of  DSS  and  the  effects  of  information  technology  on  an  organization. 

B.         WHAT  IS  DSS? 

Decision  Support  Systems  (DSS)  are  defined  by  Sprague  and  Carlson  "...  as 
interactive  computer-based  systems  that  help  decisionmakers  utilize  data  and  models  to 
solve  unstructured  problems"  [Ref  3].  The  key  points  of  this  definition  are  interactive 


10 


Decision  Focus 


DSS 


Information 
Focus 


6 


Data 
Focus 


MIS 


6 


TPS 


O 


Figure  2.  The  Notional  View  After  Ref.  [3] 

and  unstructured.  MIS  is  not  usually  interactive  and  the  reports  generated  for  the  manager 

are  usually  structured  and  repetitive.  Another  important  point  is  that  the  purpose  is  to  help 

the  decisionmaker  to  solve  the  problem.  It  is  not  to  replace  the  decisionmaker  as  in  an 

Expert  System.  Bennett's  definition  of  DSS: 

A  coherent  system  of  computer-based  technology  (hardware,  software, 
and  supporting  documentation)  used  by  managers  as  an  aid  to  their 
decisionmaking  in  semistructured  decision  tasks.  We  stress  supporting 
rather  than  replacing  managerial  judgments.  We  focus  on  improving  the 
effectiveness  of  decisionmaking  rather  than  on  merely  improving  its 
efficiency.  [Ref  4] 

Bennett  defines  when  a  task  is  considered  an  unstructured  task: 

•     Objectives  are  ambiguous  and  nonoperational,  or  objectives  are  relatively 
operational  but  numerous  and  conflicting. 
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Figure  3.  The  Theoretical  View  After  Ref.  [3] 


•  It  is  difficult  to  determine  the  cause  (after  the  fact)  of  changes  in  decision 
outcomes  and  to  predict  (in  advance)  the  effect  on  decision  outcomes  of  the 
actions  taken  by  the  decisionmaker. 

•  It  is  uncertain  what  actions  taken  by  the  decisionmaker  might  affect  decision 
outcomes.  [Ref.  4] 

COMPONENTS  OF  A  DSS 

There  are  three  components  of  a  DSS: 

•  Language  System  (LS)  -  This  system  is  used  by  the  user  to  interface  with  the 
DSS.  It  may  include  direct  retrieval  languages  and  computation  languages. 
This  allows  the  decisionmaker  to  express  commands  and  statements,  but  at  the 
same  time,  limits  the  decisionmaker  to  a  finite  number  of  expressions. 
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•  Knowledge  System  (KS)  -  This  system  contains  the  knowledge  of  this 
decisionmakers  problem  domain.  The  knowledge  must  be  organized  and  must 
be  retrievable  in  a  systematic  manner. 

•  Problem-Processing  System  (PPS)  -  The  PPS  is  a  system  which  understands 
the  decisiormiakers  statements  or  commands  and  the  representation  of  the 
knowledge  in  the  KS.  The  PPS  takes  the  relatively  simple  commands  from  the 
language  system  and  processes  them  into  the  more  complex  operations  of  the 
retrieval  system  or  the  KS  and  the  computations  required  for  the  DSS.  [Ref.  5] 

D.         THE  CHARACTERISTICS  OF  A  DSS 

Even  though  there  are  various  definitions  of  a  DSS,  usually  dependent  upon  the 
viewpoint  and  background  of  authors  in  the  field,  the  characteristics  of  a  DSS  can  be 
summarized  as  follows: 

•  DSS  assists  decisionmakers  to  deal  with  multi-level  problems  structured  and 
unstructured  [Ref  6]. 

•  DSS  supports  the  decisionmaker  with  an  adaptive  point  of  view.  In  this  point, 
a  DSS  is  better  than  a  conventional  MIS  in  that  a  MIS  caimot  adapt  itself 
easily  to  new  or  unique  situations. 

•  A  DSS  is  interactive.  The  decisionmaker  can  use  the  system  to  collect, 
process,  display,  store  and  retrieve  information  in  real  time. 

•  A  DSS  is  used  to  support  the  decisionmaker  not  replace  the  manager's 
judgment  and  experience. 

•  The  purpose  of  a  DSS  is  to  improve  the  effectiveness  of  the  decisionmaker 
and  is  not  generally  targeted  at  efficiency. 

•  A  DSS  must  be  easy  to  operate  because  it  is  designed  to  be  used  by  the 
decisionmaker  not  computer  experts  or  even  specially  trained  clerks.  The 
system  is  interactive  and  is  intended  for  the  direct  use  of  the  decisionmaker. 

•  The  DSS  must  be  adaptive.  Over  time,  the  type  and  nature  of  the  decisions 
that  the  DSS  is  used  for  will  change.  If  this  were  not  the  case,  the  DSS  might 
better  be  replaced  by  an  expert  system.  A  DSS  is  intended  to  be  used  for  one 
time  only  or  unique  decisions  that  are  unstructured. 
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•  A  DSS  must  efficiently  assist  decisionmakers  in  making  decisions.  If  the 
system  is  not  efficient  it  simply  will  not  be  used  by  managers  who  must 
budget  their  time  in  the  most  effective  manner  to  realize  the  goals  of  their 
organization. 

•  A  DSS  should  assist  in  training  inexperienced  managers  in  that  it  will  present 
to  them  information  for  a  decision  which  their  inexperience  otherwise  may 
have  led  them  to  overlook. 

E.         THE  FRAMEWORK  OF  A  DSS 

The  information  used  by  the  DSS  is  generated  from  interaction  with  the  user, 
information  from  the  database  and  model  analysis.  Figure  4  shows  the  components  of  a 
DSS  which  consists  of  five  parts. 

1.  Personnel 

There  are  three  levels  of  DSS  technology  and  five  associated  roles  for  managers 
and  technicians  in  both  the  use  and  development  of  a  DSS.  They  are  shown  in  Figure  5: 

•  Manager  or  user  -  The  person  faced  with  the  responsibility  of  the  decision. 

•  Intermediary  -  The  assistant  or  the  staff  of  the  manager  or  user  of  the  DSS. 

•  DSS  Builder  -  The  person  who  is  familiar  with  computer  systems  and  also 
familiar  with  the  problem  area  of  the  decision. 

•  Technical  Support  -  The  person  or  a  team  who  is  acquainted  with  the  problem 
area  but  whose  expertise  is  in  database,  management,  model  building  and  the 
computer  system  that  supports  the  DSS. 

•  Toolsmith  -  A  person  or  a  team  whose  responsibility  is  to  develop  new 
technologies,  software  and  hardware  to  provide  the  DSS  with  better  or  more 
complete  data  or  models.  [Ref  6] 

2.  Hardware 

Computers,  peripheral  equipment  and  facilities  for  the  maintenance  and  use  of  this 
equipment  are  required.  Telecommunication  equipment  for  remote  connections  to 
databases  and  other  information  sources  are  needed  for  large  systems.  Hardware  to 
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maintain  backups  of  the  software  and  databases  used  by  the  system  are  required  except 
for  the  simplest  systems.  Remote  storage  facilities  to  maintain  archives  of  the  systems 
software  are  necessary  for  safety. 

3.  Software 

Software  consists  of  Database  Management  Software  (DBMS),  Model  Base 
Management  Software  (MBMS)  and  dialogue  generators  for  the  interactive  element  of 
the  DSS;  compilers  and  special  software  used  to  maintain  the  system  and  the  networks 
used  by  the  system;  software  for  maintaining  archives  of  databases  and  program  sources 
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Figure  4.  Components  of  a  DSS  After  Ref.  [6] 
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Figure  5.  DSS  Levels  and  Associated  Roles  From  Ref.  [6] 

used  in  the  system  and  software  for  debugging  the  DSS;  and  diagnostic  software  for  the 
maintenance  of  the  systems  computers  and  telecommunication  facihties. 

Software  makes  up  three  subsystems  of  the  DSS,  the  dialogue  subsystem  which  is 
used  for  interacting  with  the  user,  the  data  subsystem  which  supplies  the  DSS  knowledge 
system  and  the  model  subsystem  software  which  is  used  by  the  PPS  in  calculations  and 
the  presentation  of  data  to  the  user.  Peripheral  to  this,  are  the  normal  maintenance 
software  used  to  maintain  the  data  system  and  the  hardware  of  the  DSS. 

4.  Database 

The  database  used  by  the  system  is  a  collection  of  information  necessary  for  the 
DSS  to  function.  It  must  include  software  and  hardware  required  to  maintain  any  database 
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system.  This  includes  backups  and  conversion  software  to  present  the  database  in  a  usable 
form  to  the  DSS.  It  may  include  software  required  to  access  remote  databases  across 
networks  or  other  telecommunication  links. 

5.  Model  Base 

The  model  base  is  made  up  of  standard  mathematics  and  statistic  packages  used 
by  the  DSS.  It  may  include  special  purpose  software  created  to  support  a  particular 
decision  process. 

F.         DIALOGUE  SUBSYSTEM 

Much  of  the  power,  flexibility  and  usability  characteristics  of  a  DSS  are  derived 
from  its  interface  to  the  user.  This  makes  the  dialogue  subsystem  software  the  most 
important  subsystem  in  the  DSS.  Without  a  flexible  and  very  usable  interface,  the  DSS 
will  not  be  used  by  managers  who  cannot  devote  time  to  overcoming  limitations  in  the 
user  interface.  The  dialogue  subsystem  is  itself  made  up  of  three  systems: 

•  The  Action  Language  -  The  softAvare  which  interprets  the  users  input  and 
conveys  commands  or  requests  to  the  DSS. 

•  The  Display  Language  -  The  software  which  displays  the  results  of  requests, 
commands  and  model  runs  to  the  user.  This  may  be  in  the  form  of  a  CRT 
screen,  printers  or  other  graphics  output. 

•  Knowledge  Base  -  Knowledge  base  consists  of  the  organizational  knowledge 
and  the  users  previous  inputs.  This  may  include  manuals  and  help  files  for  the 
operation  of  the  DSS.  [Ref.  4] 

Figure  6  shows  a  typical  dialogue  system. 
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Figure  6.  User/Terminal  Interface  After  Ref.  [4] 


THE  DATA  SUBSYSTEM 


The  functions  of  the  data  subsystem  are  to  query  the  database  as  a  result  of 
requests  from  the  models  and  dialogue  subsystem.  This  should  include  the  ability  to 
maintain  the  database  records  by  inserting,  deleting  and  adding  individual  records.  This 
system  should,  in  addition,  be  able  to  request  updates  to  databases  that  are  held  locally  at 
the  users  site  so  these  databases  reflect  data  whose  original  source  is  from  remote 
systems.  The  data  subsystem  is  shown  in  Figure  7 

1.  Rich  Set  of  Data  Sources 

The  data  for  the  DSS  must  come  from  external  sources  and  cannot  depend  solely 
on  local  or  internal  sources  since  the  decisions  made  by  upper  management  levels  are 
heavily  dependent  on  external  data  sources.  In  addition,  management  decisions  must  also 
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Figure  7.  The  Data  Subsystem  From  Ref.  [3] 

be  based  on  decisions  made  in  the  past.  This  requires  local  historical  data  be  presented  by 
the  DSS. 

2.  Data  Capture  and  Extraction  Process 

The  nature  of  the  DSS  requires  that  the  data  extraction  process  and  the  DBMS 
which  manages  the  database  be  flexible  and  allow  rapid  response  to  users  request  for 
data.  This  is  because  of  the  interactive  nature  of  the  DSS.  If  a  manager  spends  too  much 
time  waiting  for  a  request  to  be  satisfied  by  the  DBMS,  the  next  time  he  will  not  make  the 
request,  thereby,  bypassing  that  piece  of  information  in  making  the  decision  and 
essentially  disregarding  the  DSS  in  his  decisionmaking  process.  If  the  system  is  not 
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flexible  the  manager  may  not  even  be  able  to  request  the  information  he  requires  to  make 
the  decision,  which  will  again  result  in  the  DSS  not  being  utilized. 

H.        THE  MODEL  SUBSYSTEM 

The  most  promising  aspect  of  DSS  is  its  ability  to  integrate  data  access  with 
decision  models.  It  does  this  by  imbedding  the  model  subsystem  in  the  DSS  and 
providing  a  database  on  which  to  operate  the  models.  This  integration  provides  powerful 
what-if  scenarios  for  the  user.  Figure  8  shows  the  components  of  the  model  subsystem. 
[Ref  3] 
The  key  capabilities  provided  to  the  DSS  by  the  model  subsystem  include: 

•     The  ability  to  create  new  models  rapidly  and  easily. 


• 


The  ability  to  catalogue  and  maintain  archives  of  a  wdde  range  of  models 
supporting  all  levels  of  management. 

•  The  ability  to  access  and  integrate  models  to  create  other  models  and  use  these 
as  building  blocks  to  more  complex  simulations. 

•  The  ability  to  interrelate  these  models  Mdth  the  database. 

•  The  ability  to  manage,  maintain  and  archive  these  models  as  if  they  were 
records  in  a  database. 

I.    THE  DEVELOPMENT  APPROACH  FOR  DSS 

There  is  no  universal  decisionmaking  theory;  varying  conditions  and  rapidly 
changing  circumstance  make  a  DSS  a  good  tool  for  tracking  change.  In  order  to  track 
changing  circumstances  an  interactive  system  combining  analysis,  modeling,  data  access 
and  presentation  in  a  single  step  is  required.  An  initial  system  can  be  built  to  solve  a 
small  problem.  Once  this  system  is  capable  of  supporting  decisions  in  a  limited  area,  it 
can  be  expanded  and  improved  until  it  Mdll  support  a  v^de  variety  of  decisions  required 
by  the  organization.  DSS  are  in  a  constant  state  of  change  reflecting  new  technology,  new 
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data  sources  and  new  management  strategies.  This  in  itself  should  lead  to  a  system  which 
is  adaptive  and  flexible. 

An  adaptive  system  is  defined  as  a  system  with  three  abilities  related  to  time.  In 
the  short  term  it  must  be  flexible  enough  to  solve  a  given  problem  within  a  given  scope. 
In  the  midterm,  the  system  must  evolve  to  accommodate  changes  in  scope  and  in  the  long 
term,  it  must  accommodate  itself  in  any  given  situation. 
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Figure  8.  The  Models  Subsystem  From  Ref.  [3] 
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III.  DSS  APPLICATION  DESIGN 

This  chapter  will  cover  my  design  of  a  DSS  to  support  the  decision  to  acquire  a 
fictional  weapons  system  from  either  a  domestic  supplier  in  the  Republic  of  China  or 
from  a  group  of  international  suppliers.  Decisions  of  this  kind  involve  cost,  political, 
technology  transfer  and  support  factors.  Decisions  to  acquire  a  system  that  are  based  on 
these  factors  are  made  at  very  high  levels  of  government.  At  this  level  of  government,  a 
decision  maker  can  demand  very  detailed  and  extensive  information  from  the  bureaucracy 
below.  My  model  of  a  DSS  is  very  simple  and  includes  very  few  criteria  included  in  the 
decisionmaking  system,  but  nevertheless,  I  believe  it  is  a  very  valid  system  because  at 
this  level,  decision  makers  need  to  reduce  the  detail  involved  in  a  decision  down  to 
factors  which  will  make  their  decision  understandable  and  in  fact,  make  it  possible  to 
come  to  a  decision.  So  the  criteria  I  have  selected  for  a  decision  at  this  level  represents  a 
large  collection  of  criteria. 

A.         DESIGN  CONSIDERATIONS 

The  first  step  in  this  chapter  will  be  to  select  a  DSS  that  has  the  following 
characteristics: 

•  Simple  to  Use  -  This  system  will  be  designed  for  very  high  level  users  who  do 
not  have  the  time  to  learn  obscure  user  interfaces  or  interpret  computer  output 
which  caimot  be  grasped  immediately.  For  people  at  this  level,  the  best  system 
and  the  most  likely  to  be  used  system  is  one  that  has  a  very  simple  user 
interface  and  presents  to  them  easily  understood  results.  In  modem  computer 
terms  that  would  be  a  graphical  user  interface  where  input  is  performed  by 
clicking  and  dragging  objects  with  a  mouse.  The  normal  output  from  these 
interfaces  are  usually  colored  graphics.  Graphics  have  the  capability  of 
presenting  large  amounts  of  information  almost  instantaneously. 

•  Flexibility  -  The  nature  of  a  DSS  is  that  it  is  flexible.  It  is  required  to  present 
results  in  real  time  to  the  user  so  that  the  user  can  make  a  decision.  Different 
users  will  require  different  criteria  or  weights  to  that  criteria  and  may  require 
different  presentations  of  the  results.  The  key  ingredient  of  a  DSS  is  that  it  is 
intended  to  provide  information  to  the  user  so  that  the  user  can  make  a 
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decision.  Without  flexibility  the  DSS  may  not  be  able  to  present  data  to  the 
user  in  a  form  he  requires  or  adjust  itself  to  new  decision  criteria  that  will 
naturally  evolve  over  time.  In  addition,  a  DSS  is  designed  to  assist  in  making 
a  decision  to  a  unique  problem.  Therefore,  if  it  was  inflexible,  it  would  not  be 
used  more  than  once. 

•  Quick  Results  -  A  system  which  is  particularly  slow  in  displaying  results  may 
be  used  the  first  time  by  a  busy  manager.  However,  the  loss  of  time  and  the 
period  of  idleness  represented  by  it  will  be  considered  when  he  is  deciding  to 
use  it  again.  Depending  on  how  slow  this  system  actually  is,  he  may  not  use  it 
at  all,  and  invest  that  idle  time  in  more  traditional  methods  of  making  a 
decision.  This  problem  would  probably  not  show  up  on  a  small  system  that 
does  not  depend  on  queries  to  large  and/or  remote  databases.  If  the  DSS  and 
its  data  can  be  centrally  contained  in  a  small  computer,  quick  results  can  be 
expected.  Decisions  to  buy  floating  point  units  or  simply  better  software 
would  be  expected  to  overcome  problems  associated  with  speed.  Quick  results 
are  also  necessary  for  a  truly  interactive  system.  If  the  user  is  not  presented 
with  immediate  feedback,  he  will  not  be  able  to  rapidly  test  scenarios  and  an 
important  feature  of  a  DSS  used  in  generating  information  for  the  user  will  be 
lost. 

•  The  Ability  to  Quantify  Criteria  -  The  system  must  be  able  to  represent  criteria 
as  a  number.  For  some  criteria  this  would  be  a  simple  process,  but  usually  this 
is  only  true  of  factors  involving  cost.  This  number  scale  must  also  effectively 
represent  quantities  that  are  less  tangible,  for  example,  the  effectiveness  of  a 
weapons  system,  the  political  cost  or  risk  factors. 

•  Interactive  -  One  of  the  key  ways  for  the  user  to  simulate  the  effects  of  his 
decision  is  to  modify  the  various  factors  until  an  acceptable  result  is  achieved. 
This  process  begins  when  the  user  changes  the  model,  examines  the  results, 
and  uses  those  results  to  make  improvements  to  the  model.  This  process  can 
also  be  used  to  discover  the  critical  points  in  the  criteria  which  would  result  in 
a  different  decision  being  made.  This  feature  will  generate  better  decisions 
over  time  as  the  user  gains  a  better  understanding  of  the  criteria  or  factors  that 
affect  his  decisions.  Without  an  interactive  feature,  the  decision  maker  would 
be  presented  with  results  which  he  would  not  understand  and  in  all 
probability,  lose  confidence  in  over  time. 

The  software  package  selected  for  this  demonstration  system  is  Criterium 
Decision  Plus  made  by  Sygenex.  This  software  runs  under  the  Windows  Operating 
System  and  it  meets  the  design  considerations  listed  above.  It  is  very  easy  to  use,  and  has 


24 


a  graphics  interface.  It  can  use  one  of  two  models.  Analytical  Hierarchy  Process  (AHP)  or 
a  Simple  Multiattribute  Rating  Technique  (SMART).  It  has  output  capabilities  which  will 
show  the  user  the  most  critical  factors  in  the  decision  and  show  the  points  at  which  the 
alternatives  will  change.  The  hardware  requirements  for  the  program  are  very  ordinary 
and  can  be  met  by  any  recently  purchased  PC.  The  system  includes  a  brainstorming 
feature  which  leads  to  a  rapid  design  for  the  DSS.  The  software  includes  more  advanced 
capabilities  which  I  have  not  used  in  this  demonstration.  They  include  an  uncertainty 
capability  and  the  ability  to  generate  reports. 

This  software  operates  on  the  following  three  principles: 

•  Hierarchy  Representation  -  The  problem  is  divided  into  definable  elements. 

•  Priority  Discrimination  -  Elements  are  ranked  relative  to  one  another. 

•  Synthesis  -  Individual  judgments  are  combined  into  an  overall  rating. 

Criterium  software  allows  a  comparison  of  each  combination  of  alternatives  for  each 
criteria  and  allows  the  user  to  assign  weights  to  each  criteria.  This  software  is  extremely 
usefiil  in  organizing  and  prioritizing  multiple  alternatives  and  multiple  criteria.  The 
software  performs  a  pairwise  comparison  between  criteria  and  then  selects  the  best 
alternative  for  the  user. 

This  software  could  also  be  used  in  a  Group  Decision  Support  System  (GDSS)  but 
it  would  require  good  leadership  and  management  skills  to  bring  various  stakeholders 
into  the  process.  The  software  has  no  automatic  communication  features  (electronic  mail) 
nor  does  the  software  provide  for  database  queries  normally  associated  with  GDSS. 

B.         DESIGN  METHODOLOGY 

For  the  design  of  the  DSS,  we  did  not  have  access  to  the  Taiwan  Navy  database. 
In  order  to  make  our  system  represent  a  real  system,  we  have  developed  a  questionnaire 
and  generated  data  that  would  represent  this  database. 
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1.  Definitions 

The  initial  step  in  the  design  of  the  DSS  is  gathering  definitions  of  criteria  and  the 
problem  domain.  The  problem  domain  is  to  decide  which  one  of  four  fictitious  weapons 
systems  to  purchase.  System  A  is  a  weapons  system  produced  by  the  French,  System  B  is 
produced  by  Korea,  System  C  by  the  United  States  and  weapons  system  D  is  produced  by 
the  Republic  of  China.  These  systems  all  have  different  capabilities,  costs  and  political 
considerations. 

The  criteria  used  to  make  the  decision  was  set  to  the  feasibility  of  acquiring  the 
equipment,  the  cost  of  the  equipment,  delivery  schedule,  political  factors,  the  technology 
transfer  to  be  gained  by  the  ROC,  the  feasibility  of  acquiring  the  weapons  system  and  of 
course,  the  capabilities  of  the  weapons  system.  This  is  a  short  list  of  criteria  to  be 
considered  when  purchasing  a  weapons  system.  A  more  complete  list  would  include 
detailed  factors  involved  in  cost  and  equipment  capabilities.  The  factors  of  politics  and 
technology  transfer  probably  cannot  be  further  detailed  simply  because  they  are  subject  to 
the  decision  maker  and  would  represent  his  opinion.  Nevertheless,  the  initial  factors  were 
broken  down  into  subfactors  which  represent  a  more  realistic  level  of  detail.  The 
subfactors  were  given  decision  level  data  not  based  on  the  survey,  but  based  on  my 
experience  in  purchasing  weapons  systems.  The  definitions  for  all  the  criteria  used  in  the 
model  are  listed  in  Appendix  B. 

With  the  Criterium  software  the  first  step  is  brainstorming  which  is  quite  easily 
done  by  simply  defining  a  goal  for  the  DSS  and  creating  a  block  for  each  of  the  criteria 
listed  above  v^thout  regard  to  the  structure  of  the  decision.  The  brainstorming  window  of 
the  Criterium  software  is  shovm  in  Figure  9. 

2.  Analysis 

The  second  step  is  the  analysis  of  the  criteria  selected  in  the  first  step.  This 
requires  us  to  determine  the  relationship  or  hierarchy  of  the  criteria  in  relation  to  our  goal 
or  decision  domain.  The  analysis  of  the  criteria  that  make  up  the  decision  results  in  six 
groups,  they  are: 
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Figure  9.  Brainstorming  Window 


•  Technology  transfer  which  is  the  only  item  in  its  group.  This  criteria  indicates 
the  technical  capacity  which  will  exist  after  the  procurement  of  the  weapons 
system.  This  criteria  represents  an  advantage  for  domestic  development  and 
research. 

•  Combat  capability,  which  is  the  only  item  in  its  group.  This  criteria  represents 
the  performance  characteristics  of  the  weapons  system  evaluated  on  the  same 
numeric  scale  as  other  criteria.  It  is  a  composite  of  the  capabilities  of  the 
weapons  system.  In  a  more  realistic  DSS  this  group  would  be  a  sum  of  the 
many  subgroups  representing  the  detailed  capabilities  of  the  system. 

•  Cost,  which  is  made  up  of  two  subgroups.  The  initial  purchase  price  of  the 
weapons   system   and   its   lifetime   support   or   logistics   support   and   the 
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operational  cost  of  the  weapons  system  or  operational  support.  Logistics 
support  represents  part  support  for  maintenance  of  the  system,  the  reliability 
of  supply  (domestic  or  international),  and  the  cost  of  tooling  up  for  parts  that 
have  become  obsolete.  Operational  support  is  the  cost  of  supporting  the 
weapons  system  when  it  is  in  use.  This  includes  training,  maintenance  and  the 
personnel  to  operate  and  perform  these  functions.  This  cost  in  a  real  world 
system  would  have  to  reflect  increases  in  persoimel  for  maintenance, 
operation,  training  and  the  effects  of  the  weapons  systems  space  and  weight 
requirements  on  board  a  vessel. 

•  Risk,  which  is  made  up  of  political  considerations  and  the  delivery  time  of  the 
weapons  system.  Certainly  in  a  real  world  system  there  would  be  more 
detailed  criteria  contributing  to  the  risk  criteria,  for  example,  early 
obsolescence  of  the  system,  unsuitability  for  shipboard  use,  low  reliability, 
does  not  meet  the  specifications  as  advertised,  cost  overruns,  and/or  the 
possibility  of  non-delivery. 

•  Feasibility  of  weapons  acquisition,  which  is  the  international  political 
considerations  involved  in  acquiring  a  weapons  system. 


•  Political  considerations,  which  is  the  effect  of  political  considerations  both 
domestic  and  international  on  the  selection  of  a  weapons  system.  For  example, 
in  the  international  category  political  considerations  would  include 
internationally  recognized  embargoes  or  acquisitions  of  weapons  systems 
which  may  initiate  an  arms  race  with  a  neighboring  country.  Domestic 
considerations  would  include  high  government  level  strategies  for  purchasing 
the  weapons  systems  or  the  wishes  of  the  legislative  body  for  either  domestic 
purchase  or  foreign  purchase  of  the  weapons  system. 

The  results  of  the  analysis  are  shown  in  Figure  10. 
3.  Questionnaire  Design 

Military  procurement  is  a  complicated  issue.  A  questionnaire  to  survey  every 
aspect  of  military  procurement  would  be  prohibitively  large.  Such  a  survey  instrument 
would  be  unlikely  to  gain  an  adequate  response  or  to  be  willingly  completed.  I  have 
designed  a  questionnaire  to  obtain  the  criteria  needed  for  a  simple  decision  hierarchy.  It 
includes  eight  questions  with  the  possibility  of  the  respondent  filling  out  an  additional 
seven  questions.  The  questionnaire  is  short,  but  the  questions  have  been  designed  such 
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Figure  10.  Results  of  Analysis 

that  their  scope  will  provide  the  information  needed  to  design  the  DSS.  Additionally, 
some  of  the  questions  were  designed  to  solicit  information  outside  the  scope  of  the  DSS, 
but  indicate  the  willingness  of  the  respondent  to  use  a  DSS.  Several  questions  are 
designed  to  see  if  the  respondent  is  familiar  with  the  term  DSS. 

When  designing  a  questiormaire  for  the  development  of  the  DSS,  it  is  necessary  to 
develop  questions  which  will  return  measurements  for  the  various  factors  or  criteria  that 
are  required  for  the  operation  of  the  model  developed  in  the  analysis  step.  The  survey 
questions  are  in  Appendix  A.  Question  3,  which  has  seven  optional  questions,  and 
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Question  4  are  the  questions  which  will  be  used  to  generate  the  weights  for  the  criteria 
developed  in  the  previous  section. 

Questions  3  and  4  only  support  the  generation  of  criteria  at  the  second  level  of  the 
hierarchy,  with  the  first  level  being  the  goal  level.  The  hierarchy  is  shown  in  Figure  1 1 
and  detailed  information  can  be  found  in  Appendix  C.  The  other  levels  of  the  hierarchy 
are  based  on  my  experience  and  are  intended  to  show  a  more  realistic  hierarchy  for  the 
decision.  For  this  thesis  I  could  not  expect  to  have  the  organizational  backing  for  a  more 
extensive  survey  that  would  provide  the  depth  of  information  required  for  the  actual 
factors  at  lower  levels  in  the  decision  hierarchy.  Therefore,  I  limited  my  survey  to  the 
first  level  of  the  decision  hierarchy. 
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Figure  11.  Decision  Hierarchy 
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The  other  questions  in  the  survey  are  intended  to  create  a  common  vocabulary  and 
direct  the  respondent  to  the  subject  of  the  survey.  Naturally,  this  cannot  be  an  extensive 
survey.  My  respondents  are  senior  officers  in  transit  between  conmiands,  going  to 
military  schools  in  the  ROC. 

The  respondents  to  the  questionnaire  are  at  different  levels  of  the  procurement 
system.  Since  this  system  is  designed  for  very  high  level  management,  I  have  weighted 
the  responses  from  the  questionnaire  in  a  ratio  of  3:2:1.  The  ratio  3  is  for  respondents 
who  are  the  actual  decision  makers.  These  are  high  ranking  executives  usually  in 
headquarters  commands.  The  ratio  of  2  is  for  staff  and  other  commands  that  make 
recommendations  directly  to  the  decision  maker.  The  ratio  of  1  is  for  the  user  of  the 
weapons  system  at  a  level  of  commanding  officer  and  leaders  who  may  be  directly 
responsible  or  involved  in  the  equipment's  use.  Each  of  these  respondents  have  a 
different  view  of  the  purchase  of  a  weapons  system.  For  example,  the  user  of  the  system 
is  usually  overly  concerned  with  the  capabilities  of  the  weapon.  The  staff  concerns  are 
directed  towards  plaiming  and  support  for  systems  throughout  the  fleet.  The  decision 
maker  is  concerned  with  the  political  and  budgetary  concerns  and  must  look  to  the  long 
range  planning  for  the  organization. 

The  method  of  the  survey  included  a  recorded  tape  with  a  background  of  DSS  and 
an  explanation  of  the  survey.  Respondents  who  were  studying  in  the  National  Defense 
University  were  surveyed  collectively.  Some  other  respondents  were  contacted 
individually.  I  sent  out  262  copies  of  the  questionnaire.  I  received  179  copies.  From 
those,  I  randomly  picked  100  copies  for  statistical  analysis.  The  purpose  was  to  reduce 
the  mathematics  involved  in  calculating  statistics  of  the  results  of  the  survey. 

To  be  properly  done,  a  survey  for  the  development  of  a  DSS  should  attempt  to 
identify  all  criteria  used  by  decision  makers  in  the  domain  that  the  DSS  is  intended  to 
function.  I  have  provided  in  the  questionnaire  two  questions  which  can  be  filled  out  by 
the  respondent  if  the  criteria  listed  is  inadequate. 
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4.  Decisions  of  Criteria 

From  the  results  of  the  statistics  obtained  from  the  questionnaire,  the  importance 
of  various  criteria  on  a  decision  can  be  determined.  The  quahty  of  output  from  the  DSS  is 
directly  related  to  how  well  the  criteria  represent  the  decision  process  of  the  user.  This 
step  is  the  most  important  step  in  the  design  process.  If  there  is  some  criteria  not  covered 
by  the  DSS  or  the  definition  of  a  criteria  is  not  precise,  the  selection  of  the  criteria  will 
produce  a  system  that  is  unsound  or  produces  dubious  results.  To  increase  the  credibility 
of  my  system  and  the  likelihood  that  it  will  produce  quality  information,  I  have 
performed  three  steps: 

•  Consulted  with  decision  making  staffs  to  acquire  the  factors  or  criteria  that 
they  consider  important.  Discussed  with  them  actual  example  weapons 
procurement. 

•  Taken  into  account  the  regulations  for  procurement  in  the  ROC  and  related 
information  and  incorporated  it  into  the  demonstration  DSS. 

•  From  the  statistic  results  of  the  questionnaire,  selected  the  most  significant 
eight  factors  which  will  be  incorporated  into  the  DSS  model. 

The  results  of  the  survey  indicate  that  most  respondents  agree  that  the  six  factors 
selected  are  important  considerations  when  making  a  purchase  of  equipment.  When 
compiling  statistics  to  generate  the  actual  factors  for  each  of  these  criteria,  I  selected  at 
random  only  100  of  the  responses  to  simplify  the  process  of  computing  the  statistics. 
Question  3  was  used  to  determine  whether  the  respondents  agreed  that  these  were 
important  criteria.  Question  4  was  used  to  generate  the  weights  that  the  respondents 
believed  these  criteria  should  have.  The  six  criteria  targeted  in  the  questionnaire  are: 

•  Technology  Transfer.  The  technical  capacity  which  will  exist  domestically 
after  the  procurement  of  the  weapons  system.  The  greatest  advantage  is 
represented  by  a  purely  domestic  development  and  research  followed  by  the 
relative  willingness  of  the  source  country  to  supply  technical  expertise  and 
information. 
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Delivery  Time.  The  time  required  to  deliver  an  operational  system,  and 
possibly  the  time  to  train  personnel  in  operation  and  maintenance  of  the 
system. 

Combat  Capability.  Represents  the  performance  characteristics  of  the  weapons 
system  evaluated  on  the  same  numeric  scale  as  the  other  criteria.  This  is  a 
composite  of  the  capabilities  of  the  weapons  system. 

Feasibility  of  Weapons/Equipment  Acquisition.  The  feasibility  of  acquiring 
the  weapons  system,  this  factor  involves  trade  restrictions,  international 
embargoes,  and  domestic  budget  considerations. 

Political  Considerations.  The  sum  of  all  political  considerations  both  domestic 
and  international. 


•     Cost.  The  cost  of  both  purchasing  the  system  and  operating  it. 

5.  Assigning  Weights 

The  most  significant  problem  in  assigning  weights  is  assigning  the  weights  to 
non-numeric  criteria.  For  example,  the  only  numeric  criteria  available  in  the  list  of  six 
first  level  criteria  is  cost.  In  the  design  of  my  questionnaire  in  Question  4  the  respondents 
responded  to  the  importance  of  each  factor  as  a  percentage  of  all  factors.  Additionally, 
they  responded  to  the  combat  capability  subcriteria  in  the  same  way.  For  example, 
combat  capability  equals  100%  and  the  subfactors  are  a  portion  of  this  100%.  I  used  this 
response  to  generate  a  factor  that  represents  the  importance  of  each  item  in  the  decision. 
Further,  I  gave  more  weight  to  the  actual  decision  makers  response  in  the  ratio  of  3:2:1  as 
mentioned  in  the  previous  section.  The  sum  of  these  multipliers  and  the  response  was 
used  to  generate  an  average  weighting  from  the  survey  response.  Since  I  have  no  actual 
cost  data  to  generate  a  numerical  criteria,  I  performed  the  same  operations  for  cost  as  the 
other  five  factors.  Appendix  C  is  a  spreadsheet  listing  of  the  factors  used  in  the  model. 
The  only  factors  computed  from  the  survey  were  the  six  listed  previously.  The  remaining 
factors  in  the  model  were  generated  ad  hoc  using  my  personal  experiences  and  I  believe 
they  would  represent  the  criteria  at  that  point  in  the  model. 
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The  two  most  significant  factors  in  the  decision  from  these  calculated  results  are 
technology  transfer  and  the  combat  capability  of  the  system.  All  other  factors  have 
approximately  the  same  weight  in  the  decision  making  process 

Technology  Transfer  calculated  to  a  weight  of  .22.  This  weight  is  entered  into  the 
Criterium  software  as  22.  All  ranges  in  the  Criterium  software  are  from  0  to  100.  In  the 
Decision  Hierarchy  technology  transfer  consists  of: 

•  Knowledge  and  skills.  Intangible  skills  and  knowledge  gained  with  the  use 
and  employment  of  advanced  or  new  technologies. 

•  Hardware  and  software.  The  actual  equipment  or  software  gained  by  the  ROC. 

In  this  section  of  the  hierarchy  knowledge  and  skills  was  assigned  a  value  of  60  and 
hardware  and  software  transfer  was  assigned  a  value  of  40.  The  alternatives  are  each 
assigned  a  value  which  is  representative  of  that  source  for  knowledge  and  skills  and 
hardware  and  software  transfer.  The  hierarchy  of  technology  transfer  is  shown  in  Figure 
12. 
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Figure  12.  Technology  Transfer 
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Risk  consists  of: 

•  Delivery  Time.  Calculated  to  a  value  of  .11  from  the  survey  results.  This  is 
entered  into  the  hierarchy  as  1 1  in  a  range  of  0  to  100. 

•  Technology  Risk.  Represents  whether  the  weapons  system  is  a  finished 
technology  or  a  leading  edge  technology.  This  points  to  the  risk  that  the 
weapon  may  in  fact  be  ineffective  since  it  is  untested  in  combat. 

Risk  is  set  to  a  60/40  ratio  of  delivery  time  to  technology  risk.  Technology  risk  is  made 
up  of  technology  risk  factors  for  each  alternative.  Delivery  time  was  of  course,  a  subject 
of  the  questionnaire  and  is  set  to  11.  This  is  the  only  criterion  that  was  subject  to  the 
questioimaire  that  was  not  at  Level  2.  The  hierarchy  for  risk  criteria  is  shown  in  Figure 
13. 
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Figure  13.  Risk 


Combat  capability  is  calculated  as  .22  and  its  effect  on  the  goal  of  the  hierarchy 
was  set  to  22.  Combat  capability  is  made  up  of  three  criteria: 
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•  Mission  Effectiveness.  A  reflection  of  the  effectiveness  of  the  weapons 
system. 

•  Integrated  Logistics.  Represents  whether  the  technology  of  the  weapons 
system  fits  well  into  the  existing  systems  used  by  the  armed  forces  of  the 
ROC.  A  weapons  which  uses  unique  spares  or  supplies  would  require  unique 
logistics  to  maintain  and  use. 

•  Weapon  Availability.  The  reliability  and  maintainability  of  the  weapons 
system. 

Mission  effectiveness  is  set  to  80,  integrated  logistics  to  5  and  availability  is  set  to  1 5  as 
to  their  effect  on  combat  capability.  The  individual  values  for  integrated  logistics  and 
availability  are  determined  by  the  individual  values  for  these  factors  from  each 
alternative. 

Mission  effectiveness  is  broken  down  into  three  subcriteria: 

•  Range.  The  effective  range  of  the  weapon. 

•  Integration.  Whether  the  weapons  system  is  well  integrated  with  the  ship's 
existing  weapons  system.  Does  it  provide  extra  capability  or  duplicate  existing 
weapons. 

•  Pay  load  Capability.  Does  the  weapon  have  the  capability  of  multiple  types  of 
warheads.  Is  the  carrying  capacity  of  the  weapon  suitable  for  the  target. 

Range,  integration  and  payload  capacity  are  set  to  a  ratio  of  60:30:10  and  each  criteria 
receives  a  rating  from  the  four  alternatives. 

The  hierarchy  for  combat  capability  criteria  and  its  sub-criteria  are  shown  in 
Figure  14. 

Feasibility  of  weapons  acquisition  is  made  up  of  three  subcriteria: 

•  Financial  Worthiness  of  the  Contractor  is  the  ability  of  the  company 
producing  the  weapons  system  to  produce  the  system  even  though  it  may  not 
be  to  their  financial  advantage  to  do  so.  It  also  indicates  the  companies 
technology  expertise  to  research  and  ability  to  complete  the  project. 
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Figure  14.  Combat  Capability 


•  International  Trade  Policies.  Some  countries  may  have  restrictions  on  the 
export  of  a  particular  type  of  weapons  system,  or  there  may  be  an 
internationally  recognized  embargo  on  the  country  that  is  the  source  for  the 
weapon. 

•  Domestic  Budget  Considerations  is  whether  or  not  the  legislative  branch  of 
government  will  budget  money  for  this  particular  weapons  system. 

Financial  worthiness,  international  policies  and  budget  considerations  are  set  to  a  ratio  of 
5:80:15  and  each  receives  a  factor  from  the  four  alternatives.  Feasibility  has  a  value  of  .17 
and  is  set  to  17  for  its  effect  on  the  overall  decision.  The  hierarchy  for  feasibility  of 
weapons  acquisition  is  shown  in  Figure  15. 

Political  considerations  are  made  up  of  two  factors: 

•  Domestic  political  considerations  include  domestic  politics  involved  with 
government  contracting. 


37 


Weapons/Equipinent  Acquisition 


;j  Financial  Wofthine»t  of  >he  Contiactoi      [ — 


Featibility  of  Weapons  Acquitition  |^ 1 international  Trade  Policies 


JDoroestic  Budget  Considerations  | — 


Figure  15.  Feasibility  of  Weapons  Acquisition 


•     International  political  considerations  involve  embargoes  to  outlawed  countries 
or  sources  of  supply  which  are  not  politically  aligned  with  the  ROC. 

Political  considerations  calculated  to  a  value  of  0.08  and  are  set  to  8  in  the  model.  There 
was  small  response  to  part  G  "Other",  that  calculated  to  a  value  of  .02.  I  included  this 
response  into  political  considerations  rather  than  create  a  new  criteria.  Domestic  and 
international  political  considerations  are  set  at  a  ratio  of  80:20.  Figure  16  shows  the 
hierarchy  for  political  considerations. 
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Figure  16.  Political  Considerations 
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Cost  consists  of: 

•  Operating  Cost.  The  cost  associated  with  the  use  of  the  equipment,  for 
example,  steaming  time  required  to  train  with  the  equipment,  the  cost  resulting 
from  a  training  exercise  which  expends  ammunition,  missiles,  etc. 

•  Procurement  Cost.  Cost  of  the  equipment  at  initial  purchase  including  the 
initial  maintenance  of  the  equipment  and  the  initial  parts  support  for  the 
equipment. 

Operating  and  procurement  costs  are  set  to  an  80:20  ratio  as  to  their  effect  on  overall 
costs. 

Operating  costs  are  further  broken  down: 

•  Maintenance  Spare  Parts  Cost  is  the  cost  of  the  parts  to  maintain  the 
equipment  and  the  cost  of  preventative  maintenance  spares  to  keep  the 
equipment  at  readiness. 

•  Training  Cost  is  the  cost  of  training  maintenance  personnel  and  operations 
personnel. 

Maintenance  and  training  costs  are  set  to  a  ratio  of  80:20  and  are  represented  in  each 
alternative. 

Procurement  costs  are  further  broken  down: 

•  Research,  Development  and  Engineering  Cost  is  the  cost  of  the  development 
of  the  weapons  system  and  the  cost  of  its  production. 

•  Initial  Procurement  is  the  minimum  cost  of  acquiring  a  production  run  of  the 
equipment. 

•  Initial  Spare  Parts  and  Training  is  the  cost  of  initial  parts  support  and  the 
initial  training  of  personnel  to  operate  and  maintain  the  system. 

•  Technical  Support  is  the  cost  of  technical  personnel  to  support  the  weapons 
system  while  ROC  personnel  are  being  trained. 

These  costs  are  set  in  a  ratio  of  10:80:40:60  and  are  represented  as  individual  factors  for 
each  alternative.  The  hierarchy  for  cost  is  shown  in  Figure  1 7. 
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Figure  17.  Cost 


6.         Simulation 

The  Criterium  software  uses  a  pairwise  comparison  method  to  simulate  the 
decision  process.  A  pairwise  comparison  is  made  between  each  criteria  in  each  group  of 
the  hierarchy  to  determine  the  effect  of  that  group  on  the  next  level  of  the  hierarchy. 

When  the  simulation  is  run  with  the  criteria  values  entered  in  the  previous  section, 
it  presents  the  weapons  system  produced  by  the  Republic  of  China  as  the  best  alternative. 
The  next  best  alternative  is  the  system  produced  by  the  U.S.  With  the  Criterium  software, 
you  can  display  a  graph  which  shows  the  contribution  of  all  criteria  made  to  all  the 
alternates.  The  magnitude  of  any  alternate  indicates  its  score  and  different  colors  indicate 
the  contribution  of  each  criteria  to  that  score.  In  the  same  menu  item,  it  is  possible  to 
display  an  ideal  alternate  which  can  be  used  to  determine  whether  any  alternate 
sufficiently  meets  the  requirements  of  the  decision.  The  results  window  for  the  Criterium 
software  shown  in  Figure  18. 
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Figure  18.  Results 

The  user  of  the  system  will  naturally  question  whether  this  is  a  reasonable 
alternative.  The  Criterium  software  provides  provisions  which  will  be  covered  in  Chapter 
IV  for  analyzing  the  sensitivity  by  weights  and  the  contribution  of  various  criteria. 
Additionally,  not  covered  in  Chapter  IV,  the  Criterium  software  has  provisions  for 
tracking  the  uncertainty  in  the  assignment  of  the  weights  to  each  criteria. 

7.  Recommendations 

The  last  step  in  our  design  methodology  is  to  compile  changes  required  to  make 
the  model  perform  correctly.  There  are  two  types  of  changes:  the  first  type  are  changes  to 
the  weights  of  various  criteria  that  are  already  in  the  model  so  that  each  level  will  reflect  a 
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reasonable  decision  hierarchy.  The  second  type  are  additions  to  the  hierarchy  or  new 
connections  between  the  levels  of  the  hierarchy.  These  recommendations  will  be 
compiled  after  testing  the  model  for  reasonableness.  Step  6  and  7  of  our  design 
methodology,  shown  in  Chapter  I,  Figure  1,  will  be  covered  in  more  detail  in  Chapter  IV. 
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IV.  SIMULATION  AND  ANALYSIS 

In  order  for  a  DSS  to  be  effective,  the  user  has  to  understand  the  assumptions  and 
criteria  that  underlie  the  model  used  in  the  system.  Without  an  understanding  of  it,  the 
manager  could  not  trust  the  result  nor  could  the  manager  modify  the  model  to  achieve 
better  results.  The  first  step  in  the  process  of  achieving  good  results  from  a  DSS  is  the 
initial  design  of  the  DSS.  This  design  must  represent  the  hierarchy  of  the  decision.  This 
was  done  in  Chapter  III.  Although  this  design  lacks  sufficient  detail  for  a  real  system,  this 
detail  was  added  by  incorporating  experiences  in  procurement.  This  created  a  DSS  with 
enough  sophistication  that  in  this  chapter  will  cover  modifying  this  initial  design  into  a 
system  likely  to  be  used  in  the  real  world  in  a  real  decision. 

A  DSS  is  intended  for  use  by  the  decision  maker.  In  Chapter  II,  it  was  determined 
that  the  characteristics  of  a  DSS  require  that  it  be  easy  to  use  and  interactive.  This  chapter 
will  demonstrate  those  aspects  in  the  Criterium  software.  An  understanding  of  the  internal 
mechanisms  of  the  DSS  is  not  required  for  a  user  to  modify  the  initial  DSS  created  in 
Chapter  III.  If  this  kind  of  detail  were  needed  by  the  user,  this  system  would  likely  fail 
because  the  users  at  this  level  are  only  interested  in  the  domain  of  the  decision.  These 
high  level  users  cannot  spend  the  time  necessary  in  discovering  the  details  of  the  software 
involved  in  the  process.  The  software  meets  this  requirement  well  by  utilizing  a  graphics 
interface  and  displaying  information  about  the  process  in  a  form  easily  understood  by  the 
user.  The  software  provides  tools  so  that  the  decision  maker  can  modify  the  hierarchy  and 
the  weighting. 

A.         REVIEWING  THE  RESULTS 

A  DSS  is  intended  to  assist  the  manager  make  a  decision.  Although  the  manager 
could  be  the  designer  of  the  DSS,  it  is  unlikely  at  this  level  of  decision  making  (purchase 
weapons/equipment)  that  is  the  case.  The  best  reviewer  of  the  results  of  the  model  would 
in  fact  be  the  decision  maker.  In  the  weighting  of  the  criteria  for  the  initial  model  it  was 
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necessary  to  multiply  the  effects  of  the  survey  described  in  Chapter  III  by  a  factor  of  3. 
This  was  done  because  the  number  of  manager's  surveyed  at  this  level  (high  level 
decision  makers),  is  very  small  and  only  they  have  a  true  feel  for  the  political  climate  and 
long  range  goals  of  the  ROC  Navy.  Others  at  a  lower  level  tend  to  narrow  their  view  to 
their  own  expertise  and  problems.  This  tends  to  make  the  lower  level  decision  maker's 
view  less  pragmatic  than  that  of  the  person  who  must  champion  the  actual  decision. 

1.  The  Ideal  Alternative 

The  results  of  the  decision  are  displayed  by  the  DSS  as  a  bar  chart  showing  the 
relative  score  for  each  alternative.  These  alternatives  are  for  fictitious  weapons  system  A, 
B,  C,  D.  Figure  19  shows  the  results  of  this  initial  model.  The  best  alternative  was  a 
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Figure  19.  Initial  Results 
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weapons  system  produced  and  developed  by  the  ROC.  The  DSS  also  displays  an  ideal 
alternative.  This  alternative  is  defined  as  the  "perfect  alternative".  This  would  be  the 
alternative  that  had  the  highest  weighting  for  any  criteria.  This  is  the  simplest  check  on 
the  reasonableness  of  the  results.  Compared  against  the  ideal  solution,  we  can  ask 
ourselves  the  question  of  whether  any  solution  compares  well  to  the  best  solution.  If  the 
decision  scores  for  the  alternatives  are  a  very  low  percentage  of  the  ideal,  it  may  indicate 
an  unresolvable  conflict  in  the  criteria  or  tell  the  decision  maker  that  he  must  look  for 
better  alternatives. 

2.  Contribution  of  Criteria 

One  of  the  simplest  analysis  done  by  the  software  is  a  bar  chart  display  of  the 
contribution  of  each  criteria  to  the  decision.  This  display  is  compiled  by  level  and  shown 
in  Figure  20  and  Figure  21.  The  user  selects  the  level,  goal,  level  2,  etc.  and  displays  the 
contribution  to  the  goal  or  a  particular  criteria  at  that  level.  This  gives  the  decision  maker 
a  view  of  the  contribution  at  each  level  to  the  decision.  With  this  information,  the  user 
can  determine  whether  the  contribution  of  a  particular  criteria  has  a  reasonable 
magnitude.  The  graphics  display  at  the  goal  level  also  shows  the  user  which  criteria  have 
contributed  most  to  the  choice  of  the  final  alternative.  This  can  be  used  to  determine  the 
source  of  any  unreasonable  information  and  would  lead  the  user  down  the  path  of  the 
hierarchy  necessary  to  correct  the  model.  The  ability  to  select  different  levels  allows  the 
user  to  fine  tune  the  model  from  level  to  level  until  the  user  discovers  the  source  for  the 
unreasonable  behavior  of  the  model. 

3.  Sensitivity  Analysis 

After  the  user  has  corrected  the  gross  errors  in  the  model  using  the  contributions 
of  various  criteria  and  comparing  the  results  to  an  ideal  result,  subtle  errors  will  remain. 
These  errors  are  the  result  of  criteria  in  the  mid-levels  of  the  hierarchy  that  are  sensitive 
to  small  changes  in  the  previous  level.  In  some  cases,  this  may  be  an  exact  model  of  the 
decision  process.  In  others,  it  may  be  a  failure  in  setting  the  weights  between  various 
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criteria.  For  the  user  to  analyze  these  effects,  the  software  provides  a  sensitivity  analysis 
mechanism. 


n  Criterium  DecisionPlm  -  (AHP  £ontriMJtion»  By  DriteiiaAnalvsis  ) 


HJBfe    Edic    V'iev<-    Bixk 


I'ie    gejultj   ^sna^    tfmdow   He^j 


mmm 


.M\^ 


Contribulions  to  Weapons/EquipnienI  Acquisition  hom  LevekLevel  2 


0.33 


D-ROC 


C-U.S. 


A-France 


B-Koiea 


Criteria: 
B Technology  Tiansfer 

^Combat  Capability 

nFeadbility  of  Weapons  Acqidsiti 

HPolitical  Considerations 

HCosf 


;r  -■  T.  .-r-t^  .f  !«»■»  tt3i^m  -rr^Mo^ 


Km^yg?.^ii,-msmxmm^(^,  ^^ .  -fh 


Levd  Nante: 


L2:  Level  2 


aelp 


Figure  20.  Contribution  to  Goal 
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Figure  21.  Contribution  to  Technology  Transfer 


Sensitivity  analysis  is  defined  as  changes  in  the  weights  or  ratings  that  change  the 
preferred  alternative.  The  software  provides  a  simple  mechanism  for  the  sensitivity 
analysis.  The  user  selects  the  bar  chart  which  shows  the  results  of  the  model.  Then  the 
user  selects  the  sensitivity  analysis  display  and  places  them  in  such  a  way  that  the  user 
can  see  both  displays.  The  sensitivity  display  provides  a  pointer  which  the  user  can  move, 
and  simultaneously,  the  result  chart  will  change  to  reflect  changes  in  the  criteria  that  the 
user  is  manipulating. 
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The  sensitivity  display  lists  the  criteria  from  most  sensitive  to  least  sensitive.  This 
is  shown  in  Figure  22.  In  the  graphics  portion  of  the  display  where  ever  the  lines  for  each 
alternative  cross,  at  that  particular  point,  the  criteria  will  lead  to  a  change  in  the  displayed 
alternative.  This  mechanism  is  much  simpler  to  operate  than,  for  example,  a  system 
where  numeric  quantities  are  displayed.  Simplified  operations  of  this  kind  are  a 
requirement  for  a  DSS  in  order  to  be  used  by  higher  level  managers.  Its  immediate 
feedback  to  the  user  gives  the  user  the  ability  to  make  rapid  tests  to  determine  whether  the 
model  is  a  good  representation  of  the  decision  process. 
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Figure  22.  Sensitivity  Analysis  Window 
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Figure  23  shows  two  windows  for  sensitivity  analysis  and  the  results  display. 
With  the  display  setup  in  this  manner,  it  is  possible  to  see  the  effects  of  moving  a 
particular  criteria  to  a  critical  point  and  a  change  of  the  recommended  alternative  to  the 
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Figure  23.  Sensitivity  Analysis  and  Results  Window 

decision.  The  graphics  would  indicate  when  more  than  one  alternative  is  involved  in  the 
change  of  the  criteria's  value.  This  may  indicate  in  the  model  that  some  criteria  should 
not  in  fact  be  modeled  because  the  range  of  this  criteria  is  narrow  for  each  alternative. 
This  can  normally  be  done  by  inspection.  When  the  graph  is  displayed  many  of  the 
alternative  lines  will  cross  the  best  alternative  very  near  to  the  current  value  indicated  by 
the  cursor  for  that  criteria.  Any  single  alternative  line  crossing  the  best  alternative  may 
indicate  that  these  alternatives  are  close  together  in  value  or  it  may  indicate  that  one  of 
the  alternatives  has  been  weighted  incorrectly.  Figure  24  shows  the  effect  of  moving  the 
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Figure  24.  Critical  Point 

line  for  knowledge  and  skills  for  the  ROC  alternative  up  to  the  critical  point  where  the 
alternative  choice  changes  from  ROC  to  US. 

4.         Tradeoffs 

This  function  of  the  Criterium  software  displays  the  criteria  at  the  lowest  level  of 
the  hierarchy  as  a  ratio  to  a  selected  criteria.  For  example,  if  the  user  selects  the  criteria  in 
the  model  of  "Knowledge  And  Skills"  as  a  reference,  it  shows  a  list  where  one  unit  of 
Knowledge  And  Skills  equals  X  units  of  all  the  other  lowest  level  criteria.  This  allows  the 
decision  maker  to  directly  observe  the  reasonableness  of  the  model  by  weighing  the 
lowest  level  criteria  against  each  other  to  insure  that  they  have  the  proper  relationship. 
This  is  the  most  difficult  to  grasp  because  it's  numerical  in  nature  and  is  presented  as  a 
simple  list.  The  Tradeoff  of  Display  is  shown  in  Figure  25. 
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Figure  25.  Tradeoffs 


B. 


MODIFICATION  OF  THE  HIERARCHY  MODEL 


Modification  of  the  model  is  most  easily  performed  by  displaying  the  model 
hierarchy  and  double  clicking  on  the  block  which  represents  the  criteria  the  user  is 
interested  in  modifying.  This  will  bring  up  the  window  where  original  weights  were 
entered.  These  weights  are  relative  values  between  the  criteria  at  the  lowest  level.  These 
weights  represent  the  relative  worthiness  of  each  weapons  system  in  that  criteria.  The 
software  automatically  normalizes  these  numbers  for  the  point  in  the  level  that  the  user  is 
modifying. 
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1.  Model  Reasonableness 

The  simplest  way  to  determine  whether  the  model  is  producing  reasonable  results 
is  to  examine  the  contribution  by  each  criteria  to  the  results.  In  Figure  20  is  a  display  of 
the  results  from  the  current  model. 

An  examination  of  these  results  indicate  that  the  largest  factor  contributing  to  the 
alternative  of  the  ROC  being  selected  over  others,  is  technology  transfer.  This  reflects  the 
results  of  this  survey  which  rated  technology  transfer  and  combat  capability  as  equal 
importance  and  the  two  most  important  criteria  in  the  model.  This  is  also  a  reasonable 
assertion  for  the  ROC  in  that  technical  capabilities  gained  by  the  ROC  are  important  as 
they  are  important  for  any  country  that  wishes  to  maintain  a  strong  defense.  Since  this  is 
the  largest  contributor  to  the  selection  of  the  ROC  as  the  best  alternative  and  the  other 
alternatives  appear  graphically  to  represent  the  amount  of  technology  transfer  that  will 
take  place  to  the  ROC,  this  would  appear  to  be  reasonable  and  unlikely  to  require 
modification. 

The  next  largest  contributor  to  the  selection  of  the  ROC  is  combat  capability. 
Combat  capability  appears  to  be  equally  distributed  between  the  four  alternatives  with  the 
exception  of  France  which  has  a  better  combat  capability.  This  is  a  reasonable  result  in 
line  with  my  experiences  in  the  procurement  of  weapons  systems. 

All  other  criteria  with  the  exception  of  risk  contribute  equally  to  the  alternatives. 
Cost  in  the  case  of  the  Korean  option  is  an  advantage,  however,  the  Korean  option  is  the 
least  likely  because  of  its  effects  on  technology  transfer.  This  leaves  the  risk  criteria  to  be 
considered.  Each  option  has  a  different  degree  of  risk  associated  with  it.  But  this  is 
unlikely  to  be  modified  because  the  ROC  has  the  most  risk  associated  with  the  selection 
and  if  the  risk  were  moved  to  the  highest  value,  it  would  still  win  out  over  the  US  option. 
A  sensitivity  analysis  of  risk  indicates  that  this  is  the  most  likely  factor  to  result  in  the  US 
option  being  selected,  but  moving  the  ROC  to  a  value  of  0  (maximum  risk)  will  only 
bring  the  ROC  option  equal  to  the  US  option. 
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The  above  analysis  can  be  performed  by  selecting  one  display  (contribution  by 
criteria)  and  would  assure  the  user  that  the  results  do  indeed  reflect  a  reasonable  option. 
As  an  exercise  I  investigated  the  contributions  to  technology  transfer  from  the  next  level 
of  the  hierarchy  (Level  2).  At  this  level  the  numbers  used  for  weighting  were  generated 
by  my  experience  and  research  into  regulations  and  past  procurements  of  these  types  of 
systems. 

Level  2  is  shown  in  Figure  21.  The  contributions  to  technology  transfer  are  two 
items.  Knowledge  and  Skill  and  the  Transfer  of  the  Hardware  and  Software.  Obviously,  if 
you  buy  a  weapons  system,  you  will  receive  the  physical  aspects  of  it  (hardware  and 
software)  but  more  important  to  the  ROC  is  the  technical  knowledge  and  skills  used  to 
develop  and  maintain  this  equipment.  This  diagram  again  shows  a  realistic  view  of 
technology  transfer.  At  this  point,  we  cannot  find  anything  to  modify  in  the  model  that 
would  make  a  significant  difference  in  the  selection  of  alternatives.  A  key  attribute  of  a 
DSS  is  that  its  purpose  is  to  assist  in  making  a  decision.  Changes  made  to  the  model 
below  a  certain  threshold  would  be  counterproductive  in  that  they  could  only  serve  in  an 
attempt  to  make  the  model  produce  a  decision.  The  real  purpose  of  a  DSS  is  to  present  the 
alternatives  to  the  decision  maker  not  to  make  the  decision  for  him. 

If  we  had  found  some  relationship  displayed  to  us  graphically  which  would  appear 
imrealistic,  the  process  of  modifying  it  would  be  as  follows: 


•  Select  the  block  in  the  hierarchy  which  represents  the  criteria  that  you  wish  to 
modify. 

•  Modify  the  weighting  for  that  criteria  to  a  better  representation  of  the  model. 

•  Display  the  results  of  the  model  and  analyze  it  for  reasonableness. 

•  If  the  results  are  reasonable,  the  software  has  presented  the  user  with  a 
prioritized  list  of  the  alternatives.  This  list  is  shown  graphically  and  a  glance 
at  it  will  show  the  user  the  best  alternative  to  the  worse  alternative.  A  glance 
will  also  show  the  user  the  relative  worth  of  each  alternative. 
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•  If  the  results  are  not  reasonable  or  require  investigation,  then  analyze  the  most 
significant  contributors  to  those  results  in  an  attempt  to  discover  the  error  in 
the  model.  Then  return  to  the  first  step. 

C.         SUMMARY 

The  Criterium  software  meets  the  majority  of  user  interface  requirements  for  a 
DSS.  Graphics  displays,  simple  input  devices  and  easily  interpreted  output  are  a 
requirement  for  a  DSS  to  function  at  a  high  level  of  management.  Without  these 
attributes,  the  system  would  probably  go  unused  or  its  output  would  acquire  a  poor 
reputation. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A. 


CONCLUSIONS 


The  preceding  chapters  have  demonstrated  the  utility  of  a  DSS  for  mihtary 
procurement.  Large  scale  intricate  decisions  in  procurement  require  detailed  and  nearly 
unlimited  quantities  of  information  to  make  the  best  possible  decision.  It  is  impossible  for 
the  decision  maker  to  simplify,  enumerate  and  analyze  this  information.  Any  decision 
maker  must  be  able  to  champion  his  decisions  to  his  superiors.  To  do  this,  he  must,  of 
course  believe  that  he  has  made  a  prudent  and  reasonable  decision.  If  he  is  overwhelmed 
with  information  that  he  cannot  quantify,  it  will  be  difficult  for  him  to  believe  that  he  is 
has  covered  every  reasonable  alternative.  A  DSS  would  assist  the  decision  maker  in 
quantifying  the  information  coming  to  him.  A  DSS  would  also  provide  support  to  the 
decision  maker  in  his  belief  that  he  has  made  the  correct  decision.  Table  1  shows  the 
differences  between  decisions  made  with  the  assistance  of  a  DSS  and  more  traditional 
methods. 


Non-DSS 

DSS  Method 

Complex  problems,  dissimilar  criteria,  large 
amoimts  of  data 

Reduce  complexity,  use  dissimilar  criteria 
comparison  methods,  automate  data  processing 

Time-consuming,  laborious  staff  work 

Fast  automated  analysis 

No  time  or  data  for  analysis 

Automated  sensitivity  analysis 

Inflexible  response  to  changing  requirements 

New  insights,  flexible  data  manipulation,  system  re- 
use, continuous  process  improvement 

Expensive  acquisition  process  and  weapons 

Improved  control  and  cost  performance 

Subjective,  inconsistent  decisions  based  solely  on 
human  judgment;  subject  to  critics 

Consistent,  objective  decisions,  open  decision 
process;  focus  on  process,  not  person 

"Fighting  fires";  crisis  management 

Quality  analysis,  plannmg,  implementation 

Table  1.  Benefits  of  DSS 
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Table  1  demonstrates  the  advantages  of  using  a  DSS  for  complex  decisions.  The 
decision  maker  not  only  makes  a  more  reliable  decision  but  receives  a  better 
understanding  of  the  factors  leading  to  that  decision. 

A  computer  cannot  replace  human  judgment  or  experience.  The  primary  purpose 
of  the  DSS  is  to  present  the  decision  maker  with  information  and  analysis  to  augment 
these  human  qualities.  The  DSS  will  provide  the  user  with  analysis  and  probable  results 
which  will  allow  the  decision  maker  to  make  a  well-informed  decision. 

From  the  survey,  we  found  that  the  higher  officer's  rank  the  less  they  tended  to 
know  about  computers.  But  most  of  the  officers  surveyed  believed  that  a  working 
knowledge  of  computer  systems  would  be  necessary  in  the  future. 

We  found  that  60%  of  the  people  responding  believed  there  were  problems  with 
the  present  procurement  system  and  over  75%  of  the  respondents  thought  that  the 
procurement  system  should  be  standardized.  These  respondents  also  thought  that  there 
should  be  a  systematic  approach  to  weapons  system  selection.  Seventy-eight  percent  of 
the  respondents  believe  that  a  DSS  would  help  decision  makers,  improve  efficiency  and 
save  money.  They  also  believed  that  DSS  would  lead  to  a  more  objective  decision.  Of  the 
respondents  49%  responded  that  they  did  not  understand  what  a  DSS  is.  Of  this  49%, 
60%  were  high  ranking  officers.  This  would  point  to  a  possible  problem  in  the 
organization  accepting  DSS  solutions  for  procurement.  Further  research  and  a  much  more 
detailed  survey  would  be  required  to  reach  a  conclusion  relating  to  education  and  training 
for  people  at  various  levels  in  the  procurement  system. 

The  Criterium  software  lacks  some  mechanisms  that  would  make  it  more  usable 
in  large  organizations.  The  most  significant  problem  is  its  inability  to  communicate  with 
other  software,  for  example,  database  queries  or  an  electronic  mail  system.  Without  these, 
the  software  would  have  difficulty  in  being  accepted  as  an  organizational  tool.  This  could 
easily  be  overcome  by  simply  modifying  the  software  to  work  with  another 
organizational  tool  which  has  these  capabilities.  For  example,  modem  spreadsheets  have 
all  of  these  capabilities  and  are  commonly  used  throughout  most  large  organizations. 
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These  modifications  to  work  with  other  software  used  by  the  organization  would  also 
give  the  Criterium  software  the  ability  to  query  databases. 

The  fact  that  the  software  lacks  some  mechanisms  does  not  mean  that  it  could  not 
be  used  to  introduce  a  pilot  system  and  ultimately  sell  an  organization  on  a  larger  DSS. 
The  softAvare  will  feed  data  to  an  Excel  spreadsheet  or  text  output  to  any  other  program; 
however,  it  only  accepts  data  from  other  versions  of  itself.  It  would  be  possible  for  an 
organization  to  set  up  a  hierarchy  of  DSS'  each  feeding  a  file  to  the  one  at  the  next  higher 
level  in  the  organization  resulting  at  the  highest  level  in  a  final  decision.  The  only 
mechanism  that  could  successfully  do  this  would  be  a  network  of  computers  that  share 
files.  This  would  be  problematic  in  a  military  situation  because  of  problems  with 
classified  material  and  information. 

B.  RECOMMENDATIONS 

•  The  ROC  should  complete  its  efforts  to  standardize  the  military  procurement 
system  as  soon  as  possible.  In  doing  this,  they  should  attempt  to  establish  a 
DSS  to  support  the  decision  making  in  the  military  procurement  process. 

•  The  ROC  should  perform  a  broad  and  detailed  survey  to  establish  the  factors 
that  contribute  to  a  decision  of  military  procurement.  This  survey  would 
establish  a  case  for  a  DSS  assisted  decision  making  rather  than  the  current 
practices. 

•  Computers  have  an  increasing  presence  at  all  management  levels.  The  ROC 
should  insure  that  every  management  level  has  a  working  understanding  of 
computers  and  computer  assisted  decision  making. 

•  The  ROC  should  try  to  move  the  procurement  process  from  an  individual 
decision  to  an  organizational  decision  supported  by  analysis  and  computer 
assisted  solutions. 

C.  FUTURE  DEVELOPMENTS 

In  this  thesis,  has  emphasized  a  DSS  as  a  solution  to  military  procurement 
problems.  The  ROC  Navy's  procurement  problems  obviously  caimot  be  solved  by  the 
implementation  of  one  system.  In  order  to  meet  its  ftiture  needs,  the  Navy  requires  long- 
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range  planning  to  provide  database  and  networking  support  for  a  DSS.  These  database 
and  networking  systems  must  meet  the  needs  of  future  support  systems. 
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APPENDIX  A.  SURVEY 


1.  Current  procurement  regulation  are  sufficient  to  prevent  misconduct.  (Circle  one 
response) 

Strongly  Disagree  Disagree  Do  Not  Know  Agree  Strongly  Agree 


2.  We  should  have  a  standardize,  systematic  approach  to  the  weapons/equipment 
selection  and  acquisition. 

Strongly  Disagree  Disagree  Do  Not  Know  Agree  Strongly  Agree 


3.  Should  the  following  factors  relating  to  weapons  selection  be  included  ? 

A.  Feasibility  of  weapons/equipment  acquisition  (for  instance  the  policy  of 
the  export  country,  export  license;  etc..) 

Strongly  Disagree  Disagree  Do  Not  Know 


B.  Cost 

Strongly  Disagree 


Disagree 


C.  Delivery  Schedule 

Strongly  Disagree  Disagree 

D.  Technology  Transfer 

Strongly  Disagree  Disagree 


Do  Not  Know 


Do  Not  Know 


Do  Not  Know 


Agree 


Agree 


Agree 


Agree 


Strongly  Agree 


Strongly  Agree 


Strongly  Agree 


Strongly  Agree 


E.  General  Specifications  of  Weapons/Equipment 
Weapons  Performance 
Integrated  Logistics  (ILS) 
Risk 


Strongly  Disagree 


Disagree 


Do  Not  Know 


Agree 


Strongly  Agree 
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F.  Political  Reasons 

Strongly  Disagree  Disagree  Do  Not  Know  Agree  Strongly  Agree 

G.  Other  (List  Additional  Factors) 


4.  What  would  the  distribution  of  each  factor  as  a  weighted  value  that  would  achieve 
the  best  possible  results  for  a  selection  process?  Please  allocate  the  appropriate 
percentage  relative  to  each  factor,  all  percentages  will  sum  to  100%. 

A.  Feasibility  of  weapons/equipment  acquisition  (for  instance  the  policy  of 
the  export  country,  export  license;  etc.) 


B.  Cost 


C.  Delivery  Schedule 


D.  Technology  Transfer 
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£.  General  Specifications  of  Weapons/Equipment 


Weapons  Performance 
Integrated  Logistics  (ILS) 
Risk 


Total  =  100% 
F.  Political  Reasons 


G.  Other  (List  Additional  Factors) 


Total  =  100% 


5.  Would  a  decision  support  system  (DSS)  assist  the  decision  maker  to  make  the 
right  choice 

Strongly  Disagree  Disagree  Do  Not  Know  Agree  Strongly  Agree 


6.  If  a  DSS  is  adopted,  will  it  help  the  process  of  weapons  equipment  acquisition  to 
meet  the  following  objectives:  (Please  circle  all  that  apply) 

A.  Efficiency  related  to  time  constraints 

B.  Efficiency  related  to  cost 

C.  Efficiency  related  to  combat  effectiveness 
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D.  Objective 

E,  Other  (Please  List) 


7.  On  which  of  the  following  factors  would  the  effectiveness  of  a  DSS  rely  ?  (Please 
circle  all  that  apply) 

A.  The  accuracy  of  budget  estimates 

B.  Pre-implementation  preparatory  staff  work 

C.  Implementation  of  contract 

D.  Other  (Please  List) 


8.  You  are  familiar  with  DSS 

Strongly  Disagree  Disagree  Do  Not  Know  Agree  Strongly  Agree 
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APPENDIX  B.  RESULTS  OF  SURVEY 
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Figure  26.  Question  1. 
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Figure  27.  Question  2. 
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■  3F.  Political  Reasons 

■  3E.  General  Specifications  of  Equipment 
(USD.  Technology  Transfer 
n3C.  Delivery  Schedule 

■  3B.  Cost 
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Figure  28.  Question  3. 
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5.  Would  a  decision  support  system 
(DSS)  assist  the  decision  maker  to  make 
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Figure  29.  Question  5. 


66 


6e.  Other 


I 


6d.  Objectivity 


6c.  Efficiency 
related  to  combat 


6b.  Efficiency 
related  to  cost 


6a.  Efficiency 
related  to  time 


6.  If  a  DSS  is  adopted,  will  it  help  the 
process  of  weapons  equipment 
acquisition  to  meet  the  following 
objectives  ? 


Population  249 


20 


40 


60 


80 


100 


Figure  30.  Question  6. 
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Figure  31.  Question  7. 
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8.  Are  you  familiar  with  DSS  ? 
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Figure  32.  Question  8. 
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APPENDIX  C.  CRITERIA 


Goal  Level 

Weights    Priorities  1                     Rating  Set 

Weapons/Equipment  Acquisition 

27 

0.27 

Technology  Transfer 

11 

0.11 

Risk 

22 

0.22 

Combat  Capability 

17 

0.17 

Feasibility  of  Weapons  Acquisition 

t              10 

0.1 

Political  Considerations 

13 

0.13 

Cost 

1 

1 

1 

Level  2 

Weights 

Priorities                        Rating  Set 

Technology  Transfer 

60 

0.6  Knowledge  and  Skills 

40 

0.4 

Hardware  and  Software 

Risk 

60 

0.6 

Delivery  Time 

40 

0.4 

Technology  Risk 

Combat  Capability 

80 

0.8 

Mission  Effectiveness 

5 

0.05  Integrated  Logistics 

15 

0.15  Weapons  Availability 

Feasibility  of  Weapons  Acquisition 

5 

0.05  Financial  Worthiness  of  the  Contractor 

80 

0.8  International  Trade  Policies 

15 

0.15! Domestic  Budget  Considerations 

Political  Considerations 

80 

0.8 

Domestic  Political  Considerations 

20 

0.2 

International  Political  Considerations 

Cost 

30 

0.3 

Operating  Cost 

70 

0.7 

Procurement  Cost 

1 
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Levels 

Wieights 

Priorities                            Rating  Set 

Kncwvledge  and  Skills                         ! 

Altematives 

Hardware  and  Software 

i 

Alternatives 

Delivery  TinrB 

Altematives 

Technology  Risk 

Alternatives 

Mssion  Effectiveness 

60 

0.6 

Range 

30 

0.3 

Integration 

10 

0.1 

Payload  Capability 

Integrated  Logistics 

Altematives 

V\feapons  Availability 

Altematives 

Rnanaal  VVbrttiiness  of  the  Qxrtractor 

Altematives 

1  ntemational  Trade  Policies 

Altematives 

Domestic  Budget  Considerations 

Altematives 

Domestic  Political  Considerations 

Altematives 

International  Political  Considerations 

Altematives 

Operating  Cost 

80 

0.8  Maintenance  Spare  Parts  Cost 

20 

0.2  Training  Cost 

Procurement  Cost 

10 

0.051  Research  and  Development  and  Engineering  Cost 

80 

0.42  [Initial  Procurement 

40 

0.21  Initial  Spare  Parts  aid  Training 

60 

0.32  Technical  Support 

Lowest  Criteria 

A-France  Rating 

A-France  Priority 

Knowledge  and  Skills 

70 

0.27 

Hardware  and  Software 

85 

0.28 

Delivery  Time 

40 

0.17 

Technology  Risk 

50 

0.19 

Range 

100 

0.37 

Integration 

100 

0.42 

Payload  Capability 

90 

0.36 

Integrated  Logistics 

30 

0.16 

Weapons  Availability 

50 

0.21 

Financial  Worthiness  of  the  Contractor 

60 

0.22 

International  Trade  Policies 

60 

0.17 

Domestic  Budget  Considerations 

30 

0.11 

Domestic  Political  Considerations 

80 

0.32 

International  Political  Considerations 

80 

0.29 

Maintenance  Spare  Parts  Cost 

30 

0.13 

Training  Cost 

30 

0.11 

Research  and  Development  and  Engineering  Cost 

40 

0.16 

Initial  Procurement 

30 

0.14 

Initial  Spare  Parts  and  Training 

30 

0.11 

Technical  Support 

60 

0.23 
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Lowest  Criteria 

B-Korea  Rating 

B-Korea  Priority 

Knowledge  and  Skills 

0 

0 

Hardware  and  Software 

40 

0.13 

Delivery  Time 

100 

0.42 

Technology  Risk 

100 

0.37 

Range 

20 

0.07 

Integration 

0 

0 

Payload  Capability 

30 

0.12 

Integrated  Logistics 

30 

0.16 

Weapons  Availability 

20 

0.08 

Financial  Worthiness  of  the  Contractor 

40 

0.15 

International  Trade  Policies 

100 

0.29 

Domestic  Budget  Considerations 

100 

0.37 

Domestic  Political  Considerations 

10 

0.04 

International  Political  Considerations 

30 

0.11 

Maintenance  Spare  Parts  Cost 

90 

0.39 

Training  Cost 

80 

0.29 

Research  and  Development  and  Engineering  Cost 

100 

0.4 

Initial  Procurement 

100 

0.48 

Initial  Spare  Parts  and  Training 

60 

0.22 

Technical  Support 

20 

0.08 

Lowest  Criteria 

C-U.S.  Rating 

C-U.S.  Priority 

Knowledge  and  Skills 

85 

0.33 

Hardware  and  Software 

80 

0.26 

Delivery  Time 

80 

0.33 

Technology  Risk 

80 

0.3 

Range 

70 

0.26 

Integration 

60 

0.25 

Payload  Capability 

70 

0.28 

Integrated  Logistics 

50 

0.26 

Weapons  Availability 

100 

0.42 

Financial  Worthiness  of  the  Contractor 

80 

0.3 

International  Trade  Policies 

85 

0.25 

Domestic  Budget  Considerations 

60 

0.22 

Domestic  Political  Considerations 

60 

0.24 

International  Political  Considerations 

70 

0.25 

Maintenance  Spare  Parts  Cost 

60 

0.26 

Training  Cost 

70 

0.25 

Research  and  Development  and  Engineering  Cost 

90 

0.36 

Initial  Procurement 

60 

0.29 

Initial  Spare  Parts  and  Training 

80 

0.3 

Technical  Support 

80 

0.31 
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Lowest  Criteria 

D-ROC  Rating 

D-ROC  Priority 

Knowledge  and  Skills 

100 

0.39 

Hardware  and  Software 

100 

0.33 

Delivery  Time 

20 

0.08 

Technology  Risk 

40 

0.15 

Range 

80 

0.3 

Integration 

80 

0.33 

Payload  Capability                                                                                  60 

0.24 

Integrated  Logistics 

80 

0.42 

Weapons  Availability 

70 

0.29 

Financial  Worthiness  of  the  Contractor 

90 

0.33 

International  Trade  Policies 

100 

0.29 

Domestic  Budget  Considerations 

80 

0.3 

Domestic  Political  Considerations 

100 

0.4 

International  Political  Considerations 

100 

0.36 

Maintenance  Spare  Parts  Cost 

50 

0.22 

Training  Cost 

100 

0.36 

Research  and  Development  and  Engineering  Cost 

20 

0.08 

Initial  Procurement 

20 

0.1 

Initial  Spare  Parts  and  Training 

100 

0.37 

Technical  Support 

100 

0.38 
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APPENDIX  D.  MODEL  DEFINITION 

MODEL:  Weapons  System  Selection 

Model  Methodology:  Analytical  Hierarchy  Process 

GOAL  LEVEL: 
Weapons/Equipment  Acquisition 

Notes: 

In  choosing  a  weapons  system  source,  our  objective  is  to  maximize  the  total 

utility  of  the  weapons  system  and  minimize  the  total  cost. 

LEVEL  2: 
Technology  Transfer 

Notes: 

This  criterion  indicates  the  technical  capacity  which  will  exist 
domestically  after  the  procurement  of  the  weapons  system.  The  greatest 
advantage  is  represented  by  a  purely  domestic  development  and  research 
followed  by  the  relative  willingness  of  the  source  country  to  supply 
technical  expertise  and  information. 


Risk 


Notes: 

This  criterion  is  used  to  show  the  risk  of  cost  overruns,  schedule 

slippage,  or  the  failure  to  meet  operational  requirements,  or  possibly 

non-delivery. 
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Combat  Capability 

Notes: 

A  figure  that  represents  the  performance  characteristics  of  the  weapons 

system  evaluated  on  the  same  numeric  scale  as  the  other  criteria.  This 

is  a  composite  of  the  capabilities  of  the  weapons  system. 
Feasibility  of  Weapons  Acquisition 

Notes: 

The  feasibility  of  acquiring  weapons  system,  this  factor  involves  trade 

restrictions,  international  embargoes,  and  domestic  budget 

considerations. 
Political  Considerations 

Notes: 

This  is  the  simi  of  all  political  considerations  both  domestic  and 

international. 
Cost 

Notes: 

This  is  the  cost  of  both  purchasing  the  system  and  operating  it. 

LEVEL  3: 
Knowledge  and  Skills 

Notes: 

These  are  intangible  skills  and  knowledge  gained  with  the  use  and 
employment  of  advanced  or  new  technologies. 
Hardware  and  Software 
Notes: 

This  is  the  actual  equipment  or  software  gained  and  the  ability  to 
maintain  and  operate  gained  by  the  ROC. 
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Delivery  Time 

Notes: 

Represents  the  time  required  to  deliver  an  operational  system,  and 

possibly  the  time  to  train  personnel  in  operation  and  maintenance  of  the 

system. 
Technology  Risk 

Notes: 

This  represents  whether  the  weapon's  system  is  a  finished  technology  or 

a  leading  edge  technology.  This  points  to  the  risk  that  the  weapon  may 

in  fact,  be  ineffective  since  it  is  untested  in  combat. 
Mission  Effectiveness 

Notes: 

This  is  a  reflection  of  the  effectiveness  of  the  weapons  system. 
Integrated  Logistics 

Notes: 

This  represents  whether  the  technology  of  the  weapons  system  fits  well 

into  the  existing  systems  used  by  the  armed  forces  of  the  ROC.  A  weapons 

which  uses  unique  spares  or  supplies  would  require  unique  logistics  to 

maintain  and  use. 
Weapons  Availability 

Notes: 

This  is  a  reflection  of  the  reliability  and  maintainability  of  the 

weapons  system 
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Financial  Worthiness  of  the  Contractor 

Notes: 

This  is  the  ability  of  the  company  producing  the  weapons  system  to 

produce  the  system  even  though  it  may  not  be  to  their  financial 

advantage  to  do  so.  It  also  indicates  the  companies  technology 

expertise  to  research  and  complete  the  project. 
International  Trade  Policies 

Notes: 

Some  countries  may  have  restrictions  on  the  export  of  a  particular  type 

of  weapons  system,  or  they  maybe  an  internationally  recognized  embargo 

on  the  country  that  is  the  source  for  the  weapon. 
Domestic  Budget  Considerations 

Notes: 

Whether  or  not  the  legislative  branch  of  government  will  budget  money 

for  this  particular  weapons  system. 
Domestic  Political  Considerations 

Notes: 

Political  considerations  include  domestic  politics  involved  with 

government  contracting. . 
International  Political  Considerations 

Notes: 

International  politics  involved  with  embargoes  to  outlawed  countries  or 

sources  of  supply  which  are  not  politically  aligned  with  the  ROC 
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Operating  Cost 

Notes: 

Operating  costs  are  the  costs  associated  with  the  use  of  the  equipment 
for  example,  steaming  time  required  to  train  with  the  equipment,  the 
cost  resulting  from  a  training  exercise  which  expends  ammunition, 
missiles,  etc. 
Procurement  Cost 
Notes: 

This  is  a  cost  of  the  equipment  at  initial  purchase  including  the 
initial  maintenance  of  the  equipment  and  the  initial  parts  support  for 
the  equipment. 

LEVEL  4: 
Range 

Notes: 

The  range  of  the  weapon 
Integration 

Notes: 

This  reflects  whether  the  weapons  system  is  well  integrated  with  the 
ship's  existing  weapons  system.  Does  it  provide  extra  capability  or 
duplicate  existing  weapons  systems. 
Payload  Capability 
Notes: 

Does  the  weapon  contain  the  capability  of  multiple  types  of  warheads.  Is 
the  carrying  capacity  of  the  weapon  suitable  for  the  target. 
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Maintenance  Spare  Parts  Cost 

Notes: 

This  is  the  cost  of  the  parts  to  maintain  the  equipment  and  the  cost  of 

preventative  maintenance  spares  to  keep  the  equipment  at  readiness. 
Training  Cost 

Notes: 

The  costs  of  training  maintenance  personnel  and  operations  personnel. 
Research  and  Development  and  Engineering  Cost 

Notes: 

This  is  the  cost  of  the  development  of  the  weapons  system  and  the  cost 

of  its  production. 
Initial  Procurement 

Notes: 

This  is  the  minimum  cost  of  acquiring  a  production  run  of  the  equipment. 
Initial  Spare  Parts  and  Training 

Notes: 

This  is  the  cost  of  initial  parts  support  and  the  initial  training  of 

personnel  to  operate  and  maintain  the  system. 
Technical  Support 

Notes: 

This  is  the  cost  of  technical  persormel  to  support  the  weapons  system 

while  ROC  personnel  are  being  trained. 

ALTERNATIVES: 

A-France 
B -Korea 
C-U.S. 
D-ROC 
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